Method and apparatus for encoding video and method and apparatus for decoding video, based on hierarchical structure of coding unit

ABSTRACT

An apparatus and method for encoding video data and an apparatus and method for decoding video data are provided. The encoding method includes: splitting a current picture into at least one maximum coding unit; determining a coded depth to output an encoding result by encoding at least one split region of the at least one maximum coding unit according to operating mode of coding tool, respectively, based on a relationship among a depth of at least one coding unit of the at least one maximum coding unit, a coding tool, and an operating mode, wherein the at least one split region is generated by hierarchically splitting the at least one maximum coding unit according to depths; and outputting a bitstream including encoded video data of the coded depth, information regarding a coded depth of at least one maximum coding unit, information regarding an encoding mode, and information regarding the relationship.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application claims priority from Korean Patent Application No.10-2009-0101191, filed on Oct. 23, 2009 in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein in itsentirety by reference.

BACKGROUND

1. Field

Apparatuses and methods consistent with exemplary embodiments relate toencoding and decoding a video.

2. Description of the Related Art

As hardware for reproducing and storing high resolution or high qualityvideo content is being developed and supplied, a need for a video codecfor effectively encoding or decoding the high resolution or high qualityvideo content is increasing. In a related art video codec, a video isencoded according to a limited encoding method based on a macroblockhaving a predetermined size.

SUMMARY

One or more exemplary embodiments provide a method and apparatus forencoding a video and a method and apparatus for decoding a video in anoperating mode of a coding tool that varies according to a size of ahierarchical structured coding unit.

According to an aspect of an exemplary embodiment, there is provided amethod of encoding video data, the method including: splitting a currentpicture of the video data into at least one maximum coding unit;determining a coded depth to output a final encoding result by encodingat least one split region of the at least one maximum coding unitaccording to at least one operating mode of at least one coding tool,respectively, based on a relationship among a depth of at least onecoding unit of the at least one maximum coding unit, a coding tool, andan operating mode, wherein the at least one split region is generated byhierarchically splitting the at least one maximum coding unit accordingto depths; and outputting a bitstream including encoded video data ofthe coded depth, information regarding a coded depth of at least onemaximum coding unit, information regarding an encoding mode, andinformation regarding the relationship among the depth of the at leastone coding unit of the at least one maximum coding unit, the codingtool, and the operating mode in the at least one maximum coding unit,wherein the coding unit may be characterized by a maximum size and adepth, the depth denotes a number of times a coding unit ishierarchically split, and as a depth deepens, deeper coding unitsaccording to depths may be split from the maximum coding unit to obtainminimum coding units, wherein the depth is deepened from an upper depthto a lower depth, wherein as the depth deepens, a number of times themaximum coding unit is split increases, and a total number of possibletimes the maximum coding unit is split corresponds to a maximum depth,and wherein the maximum size and the maximum depth of the coding unitmay be predetermined. An operation mode of a coding tool for a codingunit is determined according to a depth of the coding unit.

The information regarding the relationship among the depth of the atleast one coding unit of the at least one maximum coding unit, thecoding tool, and the operating mode, may be preset in slice units, frameunits, or frame sequence units of the current picture.

The at least one coding tool for the encoding of the at least onemaximum coding unit may include at least one of quantization,transformation, intra prediction, inter prediction, motion compensation,entropy encoding, and loop filtering.

If the coding tool, an operating mode of which is determined accordingto a depth of a coding unit, is intra prediction, the operating mode mayinclude at least one intra prediction mode classified according to anumber of directions of intra prediction or may include an intraprediction mode for smoothing regions in coding units corresponding todepths and an intra prediction mode for retaining a boundary line.

If the coding tool, an operating mode of which is determined accordingto a depth of a coding unit, is inter prediction, the operating mode mayinclude an inter prediction mode according to at least one method ofdetermining a motion vector.

If the coding tool, an operating mode of which is determined accordingto a depth of a coding unit, is transformation, the operating mode mayinclude at least one transformation mode classified according to anindex of a matrix of rotational transformation.

If the coding tool, an operating mode of which is determined accordingto a depth of a coding unit, is quantization, the operating mode mayinclude at least one quantization mode classified according to whether aquantization parameter delta is to be used.

According to an aspect of another exemplary embodiment, there isprovided a method of decoding video data, the method including:receiving and parsing a bitstream including encoded video data;extracting, from the bitstream, the encoded video data, informationregarding a coded depth of at least one maximum coding unit, informationregarding an encoding mode, and information regarding a relationshipamong a depth of at least one coding unit of the at least one maximumcoding unit, a coding tool, and an operating mode; and decoding theencoded video data in the at least one maximum coding unit according toan operating mode of a coding tool matching a coding unit correspondingto at least one coded depth, based on the information regarding thecoded depth of the at least one maximum coding unit, the informationregarding the encoding mode, and the information regarding therelationship among the depth of the at least one coding unit of the atleast one maximum coding unit, the coding tool, and the operating mode,wherein the operation mode of the coding tool for a coding unit isdetermined according to the coded depth of the coding unit.

The information regarding the relationship among the depth of the atleast one coding unit of the at least one maximum coding unit, thecoding tool, and the operating mode may be extracted in slice units,frame units, or frame sequence units of the current picture.

The coding tool for the encoding of the at least one maximum coding unitmay include at least of quantization, transformation, intra prediction,inter prediction, motion compensation, entropy encoding, and loopfiltering, wherein the decoding the encoded video data may includeperforming a decoding tool corresponding to the coding tool for theencoding of the at least one maximum coding unit.

According to an aspect of another exemplary embodiment, there isprovided an apparatus for encoding video data, the apparatus including:a maximum coding unit splitter which splits a current picture of thevideo data into at least one maximum coding unit; a coding unitdeterminer which determines a coded depth to output a final encodingresult by encoding at least one split region of the at least one maximumcoding unit according to at least one operating mode of at least onecoding tools, respectively, based on a relationship among a depth of atleast one coding unit of the at least one maximum coding unit, a codingtool, and an operating mode, wherein the at least one split region isgenerated by hierarchically splitting the at least one maximum codingunit according to depths; and an output unit which outputs a bitstreamincluding encoded video data that is the final encoding result,information regarding a coded depth of the at least one maximum codingunit, information regarding an encoding mode, and information regardingthe relationship among the depth of the at least one coding unit of theat least one maximum coding unit, the coding tool, and the operatingmode in the at least one maximum coding unit. An operation mode of acoding tool for a coding unit is determined according to a depth of thecoding unit

According to an aspect of another exemplary embodiment, there isprovided an apparatus for decoding video data, the apparatus including:a receiver which receives and parses a bitstream including encoded videodata; an extractor which extracts, from the bitstream, the encoded videodata, information regarding a coded depth of at least one maximum codingunit, information regarding an encoding mode, and information regardinga relationship among a depth of at least one coding unit of the at leastone maximum coding unit, a coding tool, and an operating mode; and adecoder which decodes the encoded video data in the at least one maximumcoding unit according to an operating mode of a coding tool matching acoding unit corresponding to at least one coded depth, based on theinformation regarding the coded depth of the at least one maximum codingunit, the information regarding the encoding mode, and the informationregarding the relationship among the depth of the at least one codingunit of the at least one maximum coding unit, the coding tool, and theoperating mode, wherein the operation mode of the coding tool for acoding unit is determined according to the coded depth of the codingunit.

According to an aspect of another exemplary embodiment, there isprovided a method of decoding video data, the method including: decodingencoded video data in at least one maximum coding unit according to anoperating mode of a coding tool matching a coding unit corresponding toat least one coded depth, based on information regarding a coded depthof the at least one maximum coding unit, information regarding anencoding mode, and information regarding a relationship among a depth ofat least one coding unit of the at least one maximum coding unit, acoding tool, and an operating mode, wherein the operation mode of thecoding tool for a coding unit is determined according to the coded depthof the coding unit.

According to an aspect of another exemplary embodiment, there isprovided a computer readable recording medium having recorded thereon aprogram for executing the method of encoding video data.

According to an aspect of another exemplary embodiment, there isprovided a computer readable recording medium having recorded thereon aprogram for executing the method of decoding video data.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or other aspects will become more apparent by describingin detail exemplary embodiments with reference to the attached drawingsin which:

FIG. 1 is a block diagram of a video encoding apparatus according to anexemplary embodiment;

FIG. 2 is a block diagram of a video decoding apparatus according to anexemplary embodiment;

FIG. 3 is a diagram for describing a concept of coding units accordingto an exemplary embodiment;

FIG. 4 is a block diagram of an image encoder based on coding units,according to an exemplary embodiment;

FIG. 5 is a block diagram of an image decoder based on coding units,according to an exemplary embodiment;

FIG. 6 is a diagram illustrating deeper coding units according to depthsand partitions according to an exemplary embodiment;

FIG. 7 is a diagram for describing a relationship between a coding unitand transformation units, according to an exemplary embodiment;

FIG. 8 is a diagram for describing encoding information of coding unitscorresponding to a coded depth, according to an exemplary embodiment;

FIG. 9 is a diagram of deeper coding units according to depths,according to an exemplary embodiment;

FIGS. 10 through 12 are diagrams for describing a relationship amongcoding units, prediction units, and transformation units, according toone or more exemplary embodiments;

FIG. 13 is a diagram for describing a relationship among a coding unit,a prediction unit or a partition, and a transformation unit, accordingto encoding mode information of exemplary Table 1 below, according to anexemplary embodiment;

FIG. 14 is a flowchart illustrating a video encoding method according toan exemplary embodiment;

FIG. 15 is a flowchart illustrating a video decoding method according toan exemplary embodiment;

FIG. 16 is a block diagram of a video encoding apparatus based on acoding tool considering the size of a coding unit, according to anexemplary embodiment;

FIG. 17 is a block diagram of a video decoding apparatus based on acoding tool considering the size of a coding unit, according to anexemplary embodiment;

FIG. 18 is a diagram for describing a relationship among the size of acoding unit, a coding tool, and an operating mode, according to anexemplary embodiment;

FIG. 19 is a diagram for describing a relationship among a depth of acoding unit, a coding tool, and an operating mode, according to anexemplary embodiment;

FIG. 20 is a diagram for describing a relationship among a depth of acoding unit, a coding tool, and an operating mode, according to anexemplary embodiment;

FIG. 21 illustrates syntax of a sequence parameter set, in whichinformation regarding a relationship among a depth of a coding unit, acoding tool, and an operating mode is inserted, according to anexemplary embodiment;

FIG. 22 is a flowchart illustrating a video encoding method based on acoding tool considering the size of a coding unit, according to anexemplary embodiment; and

FIG. 23 is a flowchart illustrating a video decoding method based on acoding tool considering the size of a coding unit, according to anexemplary embodiment.

DETAILED DESCRIPTION

Hereinafter, exemplary embodiments will be described more fully withreference to the accompanying drawings. Furthermore, expressions such as“at least one of,” when preceding a list of elements, modify the entirelist of elements and do not modify the individual elements of the list.In the exemplary embodiments, “unit” may or may not refer to a unit ofsize, depending on its context. Specifically, video encoding anddecoding performed based on spatially hierarchical data units accordingto one or more exemplary embodiments will be described with reference toFIGS. 1 to 15. Also, video encoding and decoding performed in anoperating mode of a coding tool that varies according to the size of acoding unit according to one or more exemplary embodiments will bedescribed with reference to FIGS. 16 to 23.

In the following exemplary embodiments, a “coding unit” refers to eitheran encoding data unit in which image data is encoded at an encoder sideor an encoded data unit in which encoded image data is decoded at adecoder side. Also, a “coded depth” refers to a depth at which a codingunit is encoded. Hereinafter, an “image” may denote a still image for avideo or a moving image, that is, the video itself.

An apparatus and method for encoding a video and an apparatus and methodfor decoding a video according to exemplary embodiments will now bedescribed with reference to FIGS. 1 to 15.

FIG. 1 is a block diagram of a video encoding apparatus 100, accordingto an exemplary embodiment. Referring to FIG. 1, the video encodingapparatus 100 includes a maximum coding unit splitter 110, a coding unitdeterminer 120, and an output unit 130.

The maximum coding unit splitter 110 may split a current picture of animage based on a maximum coding unit for the current picture. If thecurrent picture is larger than the maximum coding unit, image data ofthe current picture may be split into at least one maximum coding unit.The maximum coding unit according to an exemplary embodiment may be adata unit having a size of 32×32, 64×64, 128×128, 256×256, etc., whereina shape of the data unit is a square having a width and height insquares of 2. The image data may be output to the coding unit determiner120 according to the at least one maximum coding unit.

A coding unit according to an exemplary embodiment may be characterizedby a maximum size and a depth. The depth denotes a number of times thecoding unit is spatially split from the maximum coding unit, and as thedepth deepens or increases, deeper coding units according to depths maybe split from the maximum coding unit to a minimum coding unit. A depthof the maximum coding unit is an uppermost depth and a depth of theminimum coding unit is a lowermost depth. Since a size of a coding unitcorresponding to each depth decreases as the depth of the maximum codingunit deepens, a coding unit corresponding to an upper depth may includea plurality of coding units corresponding to lower depths.

As described above, the image data of the current picture may be splitinto the maximum coding units according to a maximum size of the codingunit, and each of the maximum coding units may include deeper codingunits that are split according to depths. Since the maximum coding unitaccording to an exemplary embodiment is split according to depths, imagedata of a spatial domain included in the maximum coding unit may behierarchically classified according to depths.

A maximum depth and a maximum size of a coding unit, which limit thetotal number of times a height and a width of the maximum coding unitcan be hierarchically split, may be predetermined.

The coding unit determiner 120 encodes at least one split regionobtained by splitting a region of the maximum coding unit according todepths, and determines a depth to output an encoded image data accordingto the at least one split region. That is, the coding unit determiner120 determines a coded depth by encoding the image data in the deepercoding units according to depths, based on the maximum coding unit ofthe current picture, and selecting a depth having a least encodingerror. Thus, the encoded image data of the coding unit corresponding tothe determined coded depth is output to the output unit 130. Also, thecoding units corresponding to the coded depth may be regarded as encodedcoding units.

The determined coded depth and the encoded image data according to thedetermined coded depth are output to the output unit 130.

The image data in the maximum coding unit is encoded based on the deepercoding units corresponding to at least one depth equal to or below themaximum depth, and results of encoding the image data are compared basedon each of the deeper coding units. A depth having the least encodingerror may be selected after comparing encoding errors of the deepercoding units. At least one coded depth may be selected for each maximumcoding unit.

The size of the maximum coding unit is split as a coding unit ishierarchically split according to depths, and as the number of codingunits increases. Also, even if coding units correspond to a same depthin one maximum coding unit, it is determined whether to split each ofthe coding units corresponding to the same depth to a lower depth bymeasuring an encoding error of the image data of each coding unit,separately. Accordingly, even when image data is included in one maximumcoding unit, the image data is split to regions according to the depthsand the encoding errors may differ according to regions in the onemaximum coding unit, and thus the coded depths may differ according toregions in the image data. Therefore, one or more coded depths may bedetermined in one maximum coding unit, and the image data of the maximumcoding unit may be divided according to coding units of at least onecoded depth.

Accordingly, the coding unit determiner 120 may determine coding unitshaving a tree structure included in the maximum coding unit. The codingunits having a tree structure according to an exemplary embodimentinclude coding units corresponding to a depth determined to be the codeddepth, from among deeper coding units included in the maximum codingunit. A coding unit of a coded depth may be hierarchically determinedaccording to depths in the same region of the maximum coding unit, andmay be independently determined in different regions. Similarly, a codeddepth in a current region may be independently determined from a codeddepth in another region.

A maximum depth according to an exemplary embodiment is an index relatedto a number of splitting times from a maximum coding unit to a minimumcoding unit. A first maximum depth according to an exemplary embodimentmay denote a total number of splitting times from the maximum codingunit to the minimum coding unit. A second maximum depth according to anexemplary embodiment may denote a total number of depth levels from themaximum coding unit to the minimum coding unit. For example, when adepth of the maximum coding unit is 0, a depth of a coding unit in whichthe maximum coding unit is split once may be set to 1, and a depth of acoding unit in which the maximum coding unit is split twice may be setto 2. Here, if the minimum coding unit is a coding unit in which themaximum coding unit is split four times, 5 depth levels of depths 0, 1,2, 3 and 4 exist. Thus, the first maximum depth may be set to 4 and thesecond maximum depth may be set to 5.

Prediction encoding and transformation may be performed according to themaximum coding unit. The prediction encoding and the transformation arealso performed based on the deeper coding units according to a depthequal to or depths less than the maximum depth, based on the maximumcoding unit. Transformation may be performed according to a method oforthogonal transformation or integer transformation.

Since the number of deeper coding units increases whenever the maximumcoding unit is split according to depths, encoding such as theprediction encoding and the transformation is performed on all of thedeeper coding units generated as the depth deepens. For convenience ofdescription, the prediction encoding and the transformation willhereinafter be described based on a coding unit of a current depth, in amaximum coding unit.

The video encoding apparatus 100 may variably select at least one of asize and a shape of a data unit for encoding the image data. In order toencode the image data, operations, such as prediction encoding,transformation, and entropy encoding, may be performed, and at thistime, the same data unit may be used for all operations or differentdata units may be used for each operation.

For example, the video encoding apparatus 100 may select a coding unitfor encoding the image data and a data unit different from the codingunit so as to perform the prediction encoding on the image data in thecoding unit.

In order to perform prediction encoding in the maximum coding unit, theprediction encoding may be performed based on a coding unitcorresponding to a coded depth, i.e., based on a coding unit that is nolonger split to coding units corresponding to a lower depth.Hereinafter, the coding unit that is no longer split and becomes a basisunit for prediction encoding will be referred to as a prediction unit. Apartition obtained by splitting the prediction unit may include aprediction unit or a data unit obtained by splitting at least one of aheight and a width of the prediction unit.

For example, when a coding unit of 2N×2N (where N is a positive integer)is no longer split and becomes a prediction unit of 2N×2N, a size of apartition may be 2N×2N, 2N×N, N×2N, or N×N. Examples of a partition typeinclude symmetrical partitions that are obtained by symmetricallysplitting at least one of a height and a width of the prediction unit,partitions obtained by asymmetrically splitting the height or the widthof the prediction unit (such as 1:n or n:1), partitions that areobtained by geometrically splitting the prediction unit, and partitionshaving arbitrary shapes.

A prediction mode of the prediction unit may be at least one of an intramode, a inter mode, and a skip mode. For example, the intra mode or theinter mode may be performed on the partition of 2N×2N, 2N×N, N×2N, orN×N. In this case, the skip mode may be performed only on the partitionof 2N×2N. The encoding is independently performed on one prediction unitin a coding unit, thereby selecting a prediction mode having a leastencoding error.

The video encoding apparatus 100 may also perform the transformation onthe image data in a coding unit based on the coding unit for encodingthe image data and on a data unit that is different from the codingunit.

In order to perform the transformation in the coding unit, thetransformation may be performed based on a data unit having a sizesmaller than or equal to the coding unit. For example, the data unit forthe transformation may include a data unit for an intra mode and a dataunit for an inter mode.

A data unit used as a base of the transformation will hereinafter bereferred to as a transformation unit. A transformation depth indicatinga number of splitting times to reach the transformation unit bysplitting the height and the width of the coding unit may also be set inthe transformation unit. For example, in a current coding unit of 2N×2N,a transformation depth may be 0 when the size of a transformation unitis also 2N×2N, may be 1 when each of the height and width of the currentcoding unit is split into two equal parts, totally split into 4transformation units, and the size of the transformation unit is thusN×N, and may be 2 when each of the height and width of the currentcoding unit is split into four equal parts, totally split into 4²transformation units, and the size of the transformation unit is thusN/2×N/2. For example, the transformation unit may be set according to ahierarchical tree structure, in which a transformation unit of an uppertransformation depth is split into four transformation units of a lowertransformation depth according to hierarchical characteristics of atransformation depth.

Similar to the coding unit, the transformation unit in the coding unitmay be recursively split into smaller sized regions, so that thetransformation unit may be determined independently in units of regions.Thus, residual data in the coding unit may be divided according to thetransformation having the tree structure according to transformationdepths.

Encoding information according to coding units corresponding to a codeddepth uses information about the coded depth and information related toprediction encoding and transformation. Accordingly, the coding unitdeterminer 120 determines a coded depth having a least encoding errorand determines a partition type in a prediction unit, a prediction modeaccording to prediction units, and a size of a transformation unit fortransformation.

Coding units according to a tree structure in a maximum coding unit anda method of determining a partition, according to exemplary embodiments,will be described in detail below with reference to FIGS. 3 through 12.

The coding unit determiner 120 may measure an encoding error of deepercoding units according to depths by using Rate-Distortion Optimizationbased on Lagrangian multipliers.

The output unit 130 outputs the image data of the maximum coding unit,which is encoded based on the at least one coded depth determined by thecoding unit determiner 120, and information about the encoding modeaccording to the coded depth, in bitstreams.

The encoded image data may be obtained by encoding residual data of animage.

The information about the encoding mode according to the coded depth mayinclude at least one of information about the coded depth, the partitiontype in the prediction unit, the prediction mode, and the size of thetransformation unit.

The information about the coded depth may be defined by using splitinformation according to depths, which indicates whether encoding isperformed on coding units of a lower depth instead of a current depth.If the current depth of the current coding unit is the coded depth,image data in the current coding unit is encoded and output. In thiscase, the split information may be defined to not split the currentcoding unit to a lower depth. Alternatively, if the current depth of thecurrent coding unit is not the coded depth, the encoding is performed onthe coding unit of the lower depth. In this case, the split informationmay be defined to split the current coding unit to obtain the codingunits of the lower depth.

If the current depth is not the coded depth, encoding is performed onthe coding unit that is split into the coding unit of the lower depth.In this case, since at least one coding unit of the lower depth existsin one coding unit of the current depth, the encoding is repeatedlyperformed on each coding unit of the lower depth, and thus the encodingmay be recursively performed for the coding units having the same depth.

Since the coding units having a tree structure are determined for onemaximum coding unit, and information about at least one encoding mode isdetermined for a coding unit of a coded depth, information about atleast one encoding mode may be determined for one maximum coding unit.Also, a coded depth of the image data of the maximum coding unit may bedifferent according to locations since the image data is hierarchicallysplit according to depths, and thus information about the coded depthand the encoding mode may be set for the image data.

Accordingly, the output unit 130 may assign encoding information about acorresponding coded depth and an encoding mode to at least one of thecoding unit, the prediction unit, and a minimum unit included in themaximum coding unit.

The minimum unit according to an exemplary embodiment is a rectangulardata unit obtained by splitting the minimum coding unit of the lowermostdepth by 4. Alternatively, the minimum unit may be a maximum rectangulardata unit that may be included in all of the coding units, predictionunits, partition units, and transformation units included in the maximumcoding unit.

For example, the encoding information output through the output unit 130may be classified into encoding information according to coding unitsand encoding information according to prediction units. The encodinginformation according to the coding units may include the informationabout the prediction mode and the size of the partitions. The encodinginformation according to the prediction units may include informationabout an estimated direction of an inter mode, a reference image indexof the inter mode, a motion vector, a chroma component of an intra mode,and an interpolation method of the intra mode. Also, information about amaximum size of the coding unit defined according to pictures, slices,or GOPs, and information about a maximum depth may be inserted into atleast one of a Sequence Parameter Set (SPS) or a header of a bitstream.

In the video encoding apparatus 100, the deeper coding unit may be acoding unit obtained by dividing at least one of a height and a width ofa coding unit of an upper depth, which is one layer above, by two. Forexample, when the size of the coding unit of the current depth is 2N×2N,the size of the coding unit of the lower depth may be N×N. Also, thecoding unit of the current depth having the size of 2N×2N may include amaximum of 4 coding units of the lower depth.

Accordingly, the video encoding apparatus 100 may form the coding unitshaving the tree structure by determining coding units having an optimumshape and an optimum size for each maximum coding unit, based on thesize of the maximum coding unit and the maximum depth determinedconsidering characteristics of the current picture. Also, since encodingmay be performed on each maximum coding unit by using any one of variousprediction modes and transformations, an optimum encoding mode may bedetermined considering characteristics of the coding unit of variousimage sizes.

Thus, if an image having high resolution or a large amount of data isencoded in a related art macroblock, a number of macroblocks per pictureexcessively increases. Accordingly, a number of pieces of compressedinformation generated for each macroblock increases, and thus it isdifficult to transmit the compressed information and data compressionefficiency decreases. However, by using the video encoding apparatus 100according to an exemplary embodiment, image compression efficiency maybe increased since a coding unit is adjusted while consideringcharacteristics of an image and increasing a maximum size of a codingunit while considering a size of the image.

FIG. 2 is a block diagram of a video decoding apparatus 200 according toan exemplary embodiment. Referring to FIG. 2, the video decodingapparatus 200 includes a receiver 210, an image data and encodinginformation extractor 220, and an image data decoder 230. Definitions ofvarious terms, such as a coding unit, a depth, a prediction unit, and atransformation unit, and information about various encoding modes forvarious operations of the video decoding apparatus 200 are similar tothose described above with reference to FIG. 1.

The receiver 210 receives and parses a bitstream of an encoded video.The image data and encoding information extractor 220 extracts encodedimage data for each coding unit from the parsed bitstream, wherein thecoding units have a tree structure according to each maximum codingunit, and outputs the extracted image data to the image data decoder230. The image data and encoding information extractor 220 may extractinformation about a maximum size of a coding unit of a current picturefrom a header about the current picture or an SPS.

Also, the image data and encoding information extractor 220 extractsinformation about a coded depth and an encoding mode for the codingunits having a tree structure according to each maximum coding unit,from the parsed bitstream. The extracted information about the codeddepth and the encoding mode is output to the image data decoder 230.That is, the image data in a bitstream is split into the maximum codingunit so that the image data decoder 230 decodes the image data for eachmaximum coding unit.

The information about the coded depth and the encoding mode according tothe maximum coding unit may be set for information about at least onecoding unit corresponding to the coded depth, and information about anencoding mode may include information about at least one of a partitiontype of a corresponding coding unit corresponding to the coded depth, aprediction mode, and a size of a transformation unit. Also, splittinginformation according to depths may be extracted as the informationabout the coded depth.

The information about the coded depth and the encoding mode according toeach maximum coding unit extracted by the image data and encodinginformation extractor 220 is information about a coded depth and anencoding mode determined to generate a minimum encoding error when anencoder, such as a video encoding apparatus 100 according to anexemplary embodiment, repeatedly performs encoding for each deepercoding unit based on depths according to each maximum coding unit.Accordingly, the video decoding apparatus 200 may restore an image bydecoding the image data according to a coded depth and an encoding modethat generates the minimum encoding error.

Since encoding information about the coded depth and the encoding modemay be assigned to a predetermined data unit from among a correspondingcoding unit, a prediction unit, and a minimum unit, the image data andencoding information extractor 220 may extract the information about thecoded depth and the encoding mode according to the predetermined dataunits. The predetermined data units to which the same information aboutthe coded depth and the encoding mode is assigned may be the data unitsincluded in the same maximum coding unit.

The image data decoder 230 restores the current picture by decoding theimage data in each maximum coding unit based on the information aboutthe coded depth and the encoding mode according to the maximum codingunits. For example, the image data decoder 230 may decode the encodedimage data based on the extracted information about the partition type,the prediction mode, and the transformation unit for each coding unitfrom among the coding units having the tree structure included in eachmaximum coding unit. A decoding process may include a predictionincluding intra prediction and motion compensation, and an inversetransformation. Inverse transformation may be performed according to amethod of inverse orthogonal transformation or inverse integertransformation.

The image data decoder 230 may perform at least one of intra predictionand motion compensation according to a partition and a prediction modeof each coding unit, based on the information about the partition typeand the prediction mode of the prediction unit of the coding unitaccording to coded depths.

Also, the image data decoder 230 may perform inverse transformationaccording to each transformation unit in the coding unit, based on theinformation about the size of the transformation unit of the coding unitaccording to coded depths, so as to perform the inverse transformationaccording to maximum coding units.

The image data decoder 230 may determine at least one coded depth of acurrent maximum coding unit by using split information according todepths. If the split information indicates that image data is no longersplit in the current depth, the current depth is a coded depth.Accordingly, the image data decoder 230 may decode encoded data of atleast one coding unit corresponding to the each coded depth in thecurrent maximum coding unit by using at least one of the informationabout the partition type of the prediction unit, the prediction mode,and the size of the transformation unit for each coding unitcorresponding to the coded depth, and output the image data of thecurrent maximum coding unit.

For example, data units including the encoding information having thesame split information may be gathered by observing the encodinginformation set assigned for the predetermined data unit from among thecoding unit, the prediction unit, and the minimum unit, and the gathereddata units may be considered to be one data unit to be decoded by theimage data decoder 230 in the same encoding mode.

The video decoding apparatus 200 may obtain information about at leastone coding unit that generates the minimum encoding error when encodingis recursively performed for each maximum coding unit, and may use theinformation to decode the current picture. That is, the coding unitshaving the tree structure determined to be the optimum coding units ineach maximum coding unit may be decoded. Also, the maximum size of thecoding unit may be determined considering at least one of resolution andan amount of image data.

Accordingly, even if image data has high resolution and a large amountof data, the image data may be efficiently decoded and restored by usinga size of a coding unit and an encoding mode, which are adaptivelydetermined according to characteristics of the image data, andinformation about an optimum encoding mode received from an encoder.

A method of determining coding units having a tree structure, aprediction unit, and a transformation unit, according to one or moreexemplary embodiments, will now be described with reference to FIGS. 3through 13.

FIG. 3 is a diagram for describing a concept of coding units accordingto an exemplary embodiment. A size of a coding unit may be expressed inwidth×height. For example, the size of the coding unit may be 64×64,32×32, 16×16, or 8×8. A coding unit of 64×64 may be split intopartitions of 64×64, 64×32, 32×64, or 32×32, and a coding unit of 32×32may be split into partitions of 32×32, 32×16, 16×32, or 16×16, a codingunit of 16×16 may be split into partitions of 16×16, 16×8, 8×16, or 8×8,and a coding unit of 8×8 may be split into partitions of 8×8, 8×4, 4×8,or 4×4.

Referring to FIG. 3, there is exemplarily provided first video data 310with a resolution of 1920×1080 and a coding unit with a maximum size of64 and a maximum depth of 2. Furthermore, there is exemplarily providedsecond video data 320 with a resolution of 1920×1080 and a coding unitwith a maximum size of 64 and a maximum depth of 3. Also, there isexemplarily provided third video data 330 with a resolution of 352×288,and a coding unit with a maximum size of 16 and a maximum depth of 1.The maximum depth shown in FIG. 3 denotes a total number of splits froma maximum coding unit to a minimum decoding unit.

If a resolution is high or a data amount is large, a maximum size of acoding unit may be large so as to increase encoding efficiency and toaccurately reflect characteristics of an image. Accordingly, the maximumsize of the coding unit of the first and second video data 310 and 320having the higher resolution than the third video data 330 may be 64.

Since the maximum depth of the first video data 310 is 2, coding units315 of the first video data 310 may include a maximum coding unit havinga long axis size of 64, and coding units having long axis sizes of 32and 16 since depths are deepened to two layers by splitting the maximumcoding unit twice. Meanwhile, since the maximum depth of the third videodata 330 is 1, coding units 335 of the third video data 330 may includea maximum coding unit having a long axis size of 16, and coding unitshaving a long axis size of 8 since depths are deepened to one layer bysplitting the maximum coding unit once.

Since the maximum depth of the second video data 320 is 3, coding units325 of the second video data 320 may include a maximum coding unithaving a long axis size of 64, and coding units having long axis sizesof 32, 16, and 8 since the depths are deepened to 3 layers by splittingthe maximum coding unit three times. As a depth deepens, detailedinformation may be precisely expressed.

FIG. 4 is a block diagram of an image encoder 400 based on coding units,according to an exemplary embodiment. The image encoder 400 may performoperations of a coding unit determiner 120 of a video encoding apparatus100 according to an exemplary embodiment to encode image data. That is,referring to FIG. 4, an intra predictor 410 performs intra prediction oncoding units, from among a current frame 405, in an intra mode, and amotion estimator 420 and a motion compensator 425 perform interestimation and motion compensation on coding units, from among thecurrent frame 405, in an inter mode by using the current frame 405 and areference frame 495.

Data output from the intra predictor 410, the motion estimator 420, andthe motion compensator 425 is output as a quantized transformationcoefficient through a transformer 430 and a quantizer 440. The quantizedtransformation coefficient is restored as data in a spatial domainthrough an inverse quantizer 460 and an inverse transformer 470, and therestored data in the spatial domain is output as the reference frame 495after being post-processed through a deblocking unit 480 and a loopfiltering unit 490. The quantized transformation coefficient may beoutput as a bitstream 455 through an entropy encoder 450.

In order for the image encoder 400 to be applied in the video encodingapparatus 100, elements of the image encoder 400, i.e., the intrapredictor 410, the motion estimator 420, the motion compensator 425, thetransformer 430, the quantizer 440, the entropy encoder 450, the inversequantizer 460, the inverse transformer 470, the deblocking unit 480, andthe loop filtering unit 490, perform operations based on each codingunit from among coding units having a tree structure while consideringthe maximum depth of each maximum coding unit.

Specifically, the intra predictor 410, the motion estimator 420, and themotion compensator 425 determine partitions and a prediction mode ofeach coding unit from among the coding units having a tree structurewhile considering a maximum size and a maximum depth of a currentmaximum coding unit, and the transformer 430 determines the size of thetransformation unit in each coding unit from among the coding unitshaving a tree structure.

FIG. 5 is a block diagram of an image decoder 500 based on coding units,according to an exemplary embodiment. Referring to FIG. 5, a parser 510parses encoded image data to be decoded and information about encodingused for decoding from a bitstream 505. The encoded image data is outputas inverse quantized data through an entropy decoder 520 and an inversequantizer 530, and the inverse quantized data is restored to image datain a spatial domain through an inverse transformer 540.

An intra predictor 550 performs intra prediction on coding units in anintra mode with respect to the image data in the spatial domain, and amotion compensator 560 performs motion compensation on coding units inan inter mode by using a reference frame 585.

The image data in the spatial domain, which passed through the intrapredictor 550 and the motion compensator 560, may be output as arestored frame 595 after being post-processed through a deblocking unit570 and a loop filtering unit 580. Also, the image data that ispost-processed through the deblocking unit 570 and the loop filteringunit 580 may be output as the reference frame 585.

In order to decode the image data in an image data decoder 230 of avideo decoding apparatus 200 according to an exemplary embodiment, theimage decoder 500 may perform operations that are performed after theparser 510. In order for the image decoder 500 to be applied in thevideo decoding apparatus 200, elements of the image decoder 500, i.e.,the parser 510, the entropy decoder 520, the inverse quantizer 530, theinverse transformer 540, the intra predictor 550, the motion compensator560, the deblocking unit 570, and the loop filtering unit 580, performoperations based on coding units having a tree structure for eachmaximum coding unit.

Specifically, the intra predictor 550 and the motion compensator 560perform operations based on partitions and a prediction mode for each ofthe coding units having a tree structure, and the inverse transformer540 performs operations based on a size of a transformation unit foreach coding unit.

FIG. 6 is a diagram illustrating deeper coding units according todepths, and partitions, according to an exemplary embodiment.

A video encoding apparatus 100 and a video decoding apparatus 200according to exemplary embodiments use hierarchical coding units so asto consider characteristics of an image. A maximum height, a maximumwidth, and a maximum depth of coding units may be adaptively determinedaccording to the characteristics of the image, or may be differently setby a user. Sizes of deeper coding units according to depths may bedetermined according to the predetermined maximum size of the codingunit.

Referring to FIG. 6, in a hierarchical structure 600 of coding unitsaccording to an exemplary embodiment, the maximum height and the maximumwidth of the coding units are each 64, and the maximum depth is 4. Sincea depth deepens along a vertical axis of the hierarchical structure 600,a height and a width of a deeper coding unit are each split. Also, aprediction unit and partitions, which are bases for prediction encodingof each deeper coding unit, are shown along a horizontal axis of thehierarchical structure 600.

That is, a first coding unit 610 is a maximum coding unit in thehierarchical structure 600, wherein a depth is 0 and a size, i.e., aheight by width, is 64×64. The depth deepens along the vertical axis,and a second coding unit 620 having a size of 32×32 and a depth of 1, athird coding unit 630 having a size of 16×16 and a depth of 2, a fourthcoding unit 640 having a size of 8×8 and a depth of 3, and a fifthcoding unit 650 having a size of 4×4 and a depth of 4 exist. The fifthcoding unit 650 having the size of 4×4 and the depth of 4 is a minimumcoding unit.

The prediction unit and the partitions of a coding unit are arrangedalong the horizontal axis according to each depth. That is, if the firstcoding unit 610 having the size of 64×64 and the depth of 0 is aprediction unit, the prediction unit may be split into partitionsincluded in the first coding unit 610, i.e., a partition 610 having asize of 64×64, partitions 612 having a size of 64×32, partitions 614having a size of 32×64, or partitions 616 having a size of 32×32.

Similarly, a prediction unit of the second coding unit 620 having thesize of 32×32 and the depth of 1 may be split into partitions includedin the second coding unit 620, i.e., a partition 620 having a size of32×32, partitions 622 having a size of 32×16, partitions 624 having asize of 16×32, and partitions 626 having a size of 16×16.

Similarly, a prediction unit of the third coding unit 630 having thesize of 16×16 and the depth of 2 may be split into partitions includedin the third coding unit 630, i.e., a partition having a size of 16×16included in the third coding unit 630, partitions 632 having a size of16×8, partitions 634 having a size of 8×16, and partitions 636 having asize of 8×8.

Similarly, a prediction unit of the fourth coding unit 640 having thesize of 8×8 and the depth of 3 may be split into partitions included inthe fourth coding unit 640, i.e., a partition having a size of 8×8included in the fourth coding unit 640, partitions 642 having a size of8×4, partitions 644 having a size of 4×8, and partitions 646 having asize of 4×4.

The fifth coding unit 650 having the size of 4×4 and the depth of 4 isthe minimum coding unit and a coding unit of the lowermost depth. Aprediction unit of the fifth coding unit 650 is only assigned to apartition having a size of 4×4.

In order to determine the at least one coded depth of the coding unitsof the maximum coding unit 610, a coding unit determiner 120 of thevideo encoding apparatus 100 performs encoding for coding unitscorresponding to each depth included in the maximum coding unit 610.

A number of deeper coding units according to depths including data inthe same range and the same size increases as the depth deepens. Forexample, four coding units corresponding to a depth of 2 are used tocover data that is included in one coding unit corresponding to a depthof 1. Accordingly, in order to compare encoding results of the same dataaccording to depths, the coding unit corresponding to the depth of 1 andfour coding units corresponding to the depth of 2 are each encoded.

In order to perform encoding for a current depth from among the depths,a least encoding error may be selected for the current depth byperforming encoding for each prediction unit in the coding unitscorresponding to the current depth, along the horizontal axis of thehierarchical structure 600. Alternatively, the minimum encoding errormay be searched for by comparing the least encoding errors according todepths, by performing encoding for each depth as the depth deepens alongthe vertical axis of the hierarchical structure 600. A depth and apartition having the minimum encoding error in the first coding unit 610may be selected as the coded depth and a partition type of the firstcoding unit 610.

FIG. 7 is a diagram for describing a relationship between a coding unit710 and transformation units 720, according to an exemplary embodiment.

A video encoding or decoding apparatus 100 or 200 according to exemplaryembodiments encodes or decodes an image according to coding units havingsizes smaller than or equal to a maximum coding unit for each maximumcoding unit. Sizes of transformation units for transformation duringencoding may be selected based on data units that are not larger than acorresponding coding unit.

For example, in the video encoding or decoding apparatus 100 or 200, ifa size of the coding unit 710 is 64×64, transformation may be performedby using the transformation units 720 having a size of 32×32.

Also, data of the coding unit 710 having the size of 64×64 may beencoded by performing the transformation on each of the transformationunits having the size of 32×32, 16×16, 8×8, and 4×4, which are smallerthan 64×64, such that a transformation unit having the least codingerror may be selected.

FIG. 8 is a diagram for describing encoding information of coding unitscorresponding to a coded depth, according to an exemplary embodiment.Referring to FIG. 8, an output unit 130 of a video encoding apparatus100 according to an exemplary embodiment may encode and transmitinformation 800 about a partition type, information 810 about aprediction mode, and information 820 about a size of a transformationunit for each coding unit corresponding to a coded depth, as informationabout an encoding mode.

The information 800 about the partition type is information about ashape of a partition obtained by splitting a prediction unit of acurrent coding unit, wherein the partition is a data unit for predictionencoding the current coding unit. For example, a current coding unitCU_0 having a size of 2N×2N may be split into any one of a partition 802having a size of 2N×2N, a partition 804 having a size of 2N×N, apartition 806 having a size of N×2N, and a partition 808 having a sizeof N×N. Here, the information 800 about the partition type is set toindicate one of the partition 804 having a size of 2N×N, the partition806 having a size of N×2N, and the partition 808 having a size of N×N

The information 810 about the prediction mode indicates a predictionmode of each partition. For example, the information 810 about theprediction mode may indicate a mode of prediction encoding performed ona partition indicated by the information 800 about the partition type,i.e., an intra mode 812, an inter mode 814, or a skip mode 816.

The information 820 about the size of a transformation unit indicates atransformation unit to be based on when transformation is performed on acurrent coding unit. For example, the transformation unit may be a firstintra transformation unit 822, a second intra transformation unit 824, afirst inter transformation unit 826, or a second intra transformationunit 828.

An image data and encoding information extractor 220 of a video decodingapparatus 200 according to an exemplary embodiment may extract and usethe information 800, 810, and 820 for decoding, according to each deepercoding unit.

FIG. 9 is a diagram of deeper coding units according to depths,according to an exemplary embodiment.

Split information may be used to indicate a change of a depth. The splitinformation indicates whether a coding unit of a current depth is splitinto coding units of a lower depth.

Referring to FIG. 9, a prediction unit 910 for prediction encoding acoding unit 900 having a depth of 0 and a size of 2N_(—)0×2N_0 mayinclude partitions of a partition type 912 having a size of2N_(—)0×2N_(—)0, a partition type 914 having a size of 2N_(—)0×N_(—)0, apartition type 916 having a size of N_(—)0×2N_(—)0, and a partition type918 having a size of N_(—)0×N_(—)0. Though FIG. 9 only illustrates thepartition types 912 through 918 which are obtained by symmetricallysplitting the prediction unit 910, it is understood that a partitiontype is not limited thereto. For example, according to another exemplaryembodiment, the partitions of the prediction unit 910 may includeasymmetrical partitions, partitions having a predetermined shape, andpartitions having a geometrical shape.

Prediction encoding is repeatedly performed on one partition having asize of 2N_(—)0×2N_(—)0, two partitions having a size of 2N_(—)0×N_(—)0,two partitions having a size of N_(—)0×2N_(—)0, and four partitionshaving a size of N_(—)0×N_(—)0, according to each partition type. Theprediction encoding in an intra mode and an inter mode may be performedon the partitions having the sizes of 2N_(—)0×2N_(—)0, N_(—)0×2N_(—)0,2N_(—)0×N_(—)0, and N_(—)0×N_(—)0. The prediction encoding in a skipmode is performed only on the partition having the size of2N_(—)0×2N_(—)0.

Errors of encoding including the prediction encoding in the partitiontypes 912 through 918 are compared, and the least encoding error isdetermined among the partition types. If an encoding error is smallestin one of the partition types 912 through 916, the prediction unit 910may not be split into a lower depth.

For example, if the encoding error is the smallest in the partition type918, a depth is changed from 0 to 1 to split the partition type 918 inoperation 920, and encoding is repeatedly performed on coding units 930having a depth of 2 and a size of N_(—)0×N_(—)0 to search for a minimumencoding error.

A prediction unit 940 for prediction encoding the coding unit 930 havinga depth of 1 and a size of 2N_(—)1×2N_(—)1 (=N_(—)0×N_(—)0) may includepartitions of a partition type 942 having a size of 2N_(—)1×2N_(—)1, apartition type 944 having a size of 2N_(—)1×N_(—)1, a partition type 946having a size of N_(—)1×2N_(—)1, and a partition type 948 having a sizeof N_(—)1×N_(—)1.

As an example, if an encoding error is the smallest in the partitiontype 948, a depth is changed from 1 to 2 to split the partition type 948in operation 950, and encoding is repeatedly performed on coding units960, which have a depth of 2 and a size of N_(—)2×N_(—)2 to search for aminimum encoding error.

When a maximum depth is d, split operations according to each depth maybe performed up to when a depth becomes d-1, and split information maybe encoded as up to when a depth is one of 0 to d-2. For example, whenencoding is performed up to when the depth is d-1 after a coding unitcorresponding to a depth of d-2 is split in operation 970, a predictionunit 990 for prediction encoding a coding unit 980 having a depth of d-1and a size of 2N (d-1)×2N (d-1) may include partitions of a partitiontype 992 having a size of 2N (d-1)×2N (d-1), a partition type 994 havinga size of 2N_(d-1)×N_(d-1), a partition type 996 having a size ofN_(d-1)×2N_(d-1), and a partition type 998 having a size ofN_(d-1)×N_(d-1).

Prediction encoding may be repeatedly performed on one partition havinga size of 2N_(d-1)×2N (d-1), two partitions having a size of2N_(d-1)×N_(d-1), two partitions having a size of N_(d-1)×2N_(d-1), fourpartitions having a size of N_(d-1)×N_(d-1) from among the partitiontypes 992 through 998 to search for a partition type having a minimumencoding error.

Even when the partition type 998 has the minimum encoding error, since amaximum depth is d, a coding unit CU_(d-1) having a depth of d-1 is nolonger split to a lower depth. In this case, a coded depth for thecoding units of a current maximum coding unit 900 is determined to bed-1 and a partition type of the current maximum coding unit 900 may bedetermined to be N_(d-1)×N_(d-1). Also, since the maximum depth is d anda minimum coding unit 980 having a lowermost depth of d-1 is no longersplit to a lower depth, split information for the minimum coding unit980 is not set.

A data unit 999 may be a minimum unit for the current maximum codingunit. A minimum unit according to an exemplary embodiment may be arectangular data unit obtained by splitting a minimum coding unit 980 by4. By performing the encoding repeatedly, a video encoding apparatus 100according to an exemplary embodiment may select a depth having the leastencoding error by comparing encoding errors according to depths of thecoding unit 900 to determine a coded depth, and set a correspondingpartition type and a prediction mode as an encoding mode of the codeddepth.

As such, the minimum encoding errors according to depths are compared inall of the depths of 1 through d, and a depth having the least encodingerror may be determined as a coded depth. The coded depth, the partitiontype of the prediction unit, and the prediction mode may be encoded andtransmitted as information about an encoding mode. Also, since a codingunit is split from a depth of 0 to a coded depth, split information ofthe coded depth is set to 0, and split information of depths excludingthe coded depth is set to 1.

An image data and encoding information extractor 220 of a video decodingapparatus 200 according to an exemplary embodiment may extract and usethe information about the coded depth and the prediction unit of thecoding unit 900 to decode the partition 912. The video decodingapparatus 200 may determine a depth, in which split information is 0 asa coded depth by using split information according to depths, and useinformation about an encoding mode of the corresponding depth fordecoding.

FIGS. 10 through 12 are diagrams for describing a relationship betweencoding units 1010, prediction units 1060, and transformation units 1070,according to one or more exemplary embodiments.

Referring to FIG. 10, the coding units 1010 are coding units having atree structure, corresponding to coded depths determined by a videoencoding apparatus 100 according to an exemplary embodiment, in amaximum coding unit. Referring to FIGS. 11 and 12, the prediction units1060 are partitions of prediction units of each of the coding units1010, and the transformation units 1070 are transformation units of eachof the coding units 1010.

When a depth of a maximum coding unit is 0 in the coding units 1010,depths of coding units 1012 and 1054 are 1, depths of coding units 1014,1016, 1018, 1028, 1050, and 1052 are 2, depths of coding units 1020,1022, 1024, 1026, 1030, 1032, and 1048 are 3, and depths of coding units1040, 1042, 1044, and 1046 are 4.

In the prediction units 1060, some coding units 1014, 1016, 1022, 1032,1048, 1050, 1052, and 1054 are obtained by splitting coding units of thecoding units 1010. In particular, partition types in the coding units1014, 1022, 1050, and 1054 have a size of 2N×N, partition types in thecoding units 1016, 1048, and 1052 have a size of N×2N, and a partitiontype of the coding unit 1032 has a size of N×N. Prediction units andpartitions of the coding units 1010 are smaller than or equal to eachcoding unit.

Transformation or inverse transformation is performed on image data ofthe coding unit 1052 in the transformation units 1070 in a data unitthat is smaller than the coding unit 1052. Also, the coding units 1014,1016, 1022, 1032, 1048, 1050, and 1052 of the transformation units 1070are different from those of the prediction units 1060 in terms of sizesand shapes. That is, the video encoding and decoding apparatuses 100 and200 according to exemplary embodiments may perform intra prediction,motion estimation, motion compensation, transformation, and inversetransformation individually on a data unit in the same coding unit.

Accordingly, encoding is recursively performed on each of coding unitshaving a hierarchical structure in each region of a maximum coding unitto determine an optimum coding unit, and thus coding units having arecursive tree structure may be obtained. Encoding information mayinclude split information about a coding unit, information about apartition type, information about a prediction mode, and informationabout a size of a transformation unit. Exemplary table 1 shows theencoding information that may be set by the video encoding and decodingapparatuses 100 and 200.

TABLE 1 Split Information 0 (Encoding on Coding Unit having Size of 2N ×2N and Current Depth of d) Size of Transformation Unit Split SplitPartition Type Information 0 Information 1 Symmetrical Asymmetrical ofof Prediction Partition Partition Transformation Transformation SplitMode Type Type Unit Unit Information 1 Intra 2N × 2N 2N × nU 2N × 2N N ×N Repeatedly Inter 2N × N 2N × nD (Symmetrical Encode Skip N × 2N nL ×2N Type) Coding (Only N × N nR × 2N N/2 × N/2 Units 2N × 2N)(Asymmetrical having Type) Lower Depth of d + 1

An output unit 130 of the video encoding apparatus 100 may output theencoding information about the coding units having a tree structure, andan image data and encoding information extractor 220 of the videodecoding apparatus 200 may extract the encoding information about thecoding units having a tree structure from a received bitstream.

Split information indicates whether a current coding unit is split intocoding units of a lower depth. If split information of a current depth dis 0, a depth in which a current coding unit is no longer split into alower depth is a coded depth. Information about a partition type,prediction mode, and a size of a transformation unit may be defined forthe coded depth. If the current coding unit is further split accordingto the split information, encoding is independently performed on splitcoding units of a lower depth.

A prediction mode may be one of an intra mode, an inter mode, and a skipmode. The intra mode and the inter mode may be defined in all partitiontypes, and the skip mode may be defined in only a partition type havinga size of 2N×2N.

The information about the partition type may indicate symmetricalpartition types having sizes of 2N×2N, 2N×N, N×2N, and N×N, which areobtained by symmetrically splitting a height or a width of a predictionunit, and asymmetrical partition types having sizes of 2N×nU, 2N×nD,nL×2N, and nR×2N, which are obtained by asymmetrically splitting theheight or the width of the prediction unit. The asymmetrical partitiontypes having the sizes of 2N×nU and 2N×nD may be respectively obtainedby splitting the height of the prediction unit in ratios of 1:3 and 3:1,and the asymmetrical partition types having the sizes of nL×2N and nR×2Nmay be respectively obtained by splitting the width of the predictionunit in ratios of 1:3 and 3:1

The size of the transformation unit may be set to be two types in theintra mode and two types in the inter mode. For example, if splitinformation of the transformation unit is 0, the size of thetransformation unit may be 2N×2N, which is the size of the currentcoding unit. If split information of the transformation unit is 1, thetransformation units may be obtained by splitting the current codingunit. Also, if a partition type of the current coding unit having thesize of 2N×2N is a symmetrical partition type, a size of atransformation unit may be N×N, and if the partition type of the currentcoding unit is an asymmetrical partition type, the size of thetransformation unit may be N/2×N/2.

The encoding information about coding units having a tree structure mayinclude at least one of a coding unit corresponding to a coded depth, acoding unit corresponding to a prediction unit, and a coding unitcorresponding to a minimum unit. The coding unit corresponding to thecoded depth may include at least one of a prediction unit and a minimumunit including the same encoding information.

Accordingly, it is determined whether adjacent data units are includedin the same coding unit corresponding to the coded depth by comparingencoding information of the adjacent data units. Also, a correspondingcoding unit corresponding to a coded depth is determined by usingencoding information of a data unit, and thus a distribution of codeddepths in a maximum coding unit may be determined.

Accordingly, if a current coding unit is predicted based on encodinginformation of adjacent data units, encoding information of data unitsin deeper coding units adjacent to the current coding unit may bedirectly referred to and used. However, it is understood that anotherexemplary embodiment is not limited thereto. For example, according toanother exemplary embodiment, if a current coding unit is predictedbased on encoding information of adjacent data units, data unitsadjacent to the current coding unit are searched using encodedinformation of the data units, and the searched adjacent coding unitsmay be referred for predicting the current coding unit.

FIG. 13 is a diagram for describing a relationship between a codingunit, a prediction unit or a partition, and a transformation unit,according to encoding mode information of exemplary Table 1, accordingto an exemplary embodiment.

Referring to FIG. 13, a maximum coding unit 1300 includes coding units1302, 1304, 1306, 1312, 1314, 1316, and 1318 of coded depths. Here,since the coding unit 1318 is a coding unit of a coded depth, splitinformation may be set to 0. Information about a partition type of thecoding unit 1318 having a size of 2N×2N may be set to be one of apartition type 1322 having a size of 2N×2N, a partition type 1324 havinga size of 2N×N, a partition type 1326 having a size of N×2N, a partitiontype 1328 having a size of N×N, a partition type 1332 having a size of2N×nU, a partition type 1334 having a size of 2N×nD, a partition type1336 having a size of nL×2N, and a partition type 1338 having a size ofnR×2N.

When the partition type is set to be symmetrical, i.e., the partitiontype 1322, 1324, 1326, or 1328, a transformation unit 1342 having a sizeof 2N×2N is set if split information (TU size flag) of a transformationunit is 0, and a transformation unit 1344 having a size of N×N is set ifa TU size flag is 1.

When the partition type is set to be asymmetrical, i.e., the partitiontype 1332, 1334, 1336, or 1338, a transformation unit 1352 having a sizeof 2N×2N is set if a TU size flag is 0, and a transformation unit 1354having a size of N/2×N/2 is set if a TU size flag is 1.

Referring to FIG. 13, the TU size flag is a flag having a value or 0 or1, though it is understood that the TU size flag is not limited to 1bit, and a transformation unit may be hierarchically split having a treestructure while the TU size flag increases from 0.

In this case, the size of a transformation unit that has been actuallyused may be expressed by using a TU size flag of a transformation unit,according to an exemplary embodiment, together with a maximum size andminimum size of the transformation unit. According to an exemplaryembodiment, a video encoding apparatus 100 is capable of encodingmaximum transformation unit size information, minimum transformationunit size information, and a maximum TU size flag. The result ofencoding the maximum transformation unit size information, the minimumtransformation unit size information, and the maximum TU size flag maybe inserted into an SPS. According to an exemplary embodiment, a videodecoding apparatus 200 may decode video by using the maximumtransformation unit size information, the minimum transformation unitsize information, and the maximum TU size flag.

For example, if the size of a current coding unit is 64×64 and a maximumtransformation unit size is 32×32, the size of a transformation unit maybe 32×32 when a TU size flag is 0, may be 16×16 when the TU size flag is1, and may be 8×8 when the TU size flag is 2.

As another example, if the size of the current coding unit is 32×32 anda minimum transformation unit size is 32×32, the size of thetransformation unit may be 32×32 when the TU size flag is 0. Here, theTU size flag cannot be set to a value other than 0, since the size ofthe transformation unit cannot be less than 32×32.

As another example, if the size of the current coding unit is 64×64 anda maximum TU size flag is 1, the TU size flag may be 0 or 1. Here, theTU size flag cannot be set to a value other than 0 or 1.

Thus, if it is defined that the maximum TU size flag isMaxTransformSizeIndex, a minimum transformation unit size isMinTransformSize, and a transformation unit size is RootTuSize when theTU size flag is 0, a current minimum transformation unit sizeCurrMinTuSize that can be determined in a current coding unit may bedefined by Equation (1):

CurrMinTuSize=max(MinTransformSize, RootTuSize/(2̂MaxTransformSizeIndex))  (1).

Compared to the current minimum transformation unit size CurrMinTuSizethat can be determined in the current coding unit, a transformation unitsize RootTuSize when the TU size flag is 0 may denote a maximumtransformation unit size that can be selected in the system. In Equation(1), RootTuSize/(2̂MaxTransformSizeIndex) denotes a transformation unitsize when the transformation unit size RootTuSize, when the TU size flagis 0, is split a number of times corresponding to the maximum TU sizeflag. Furthermore, MinTransformSize denotes a minimum transformationsize. Thus, a smaller value from amongRootTuSize/(2̂MaxTransformSizeIndex) and MinTransformSize may be thecurrent minimum transformation unit size CurrMinTuSize that can bedetermined in the current coding unit.

According to an exemplary embodiment, the maximum transformation unitsize RootTuSize may vary according to the type of a prediction mode.

For example, if a current prediction mode is an inter mode, thenRootTuSize may be determined by using Equation (2) below. In Equation(2), MaxTransformSize denotes a maximum transformation unit size, andPUSize denotes a current prediction unit size.

RootTuSize=min(MaxTransformSize, PUSize) (2).

That is, if the current prediction mode is the inter mode, thetransformation unit size RootTuSize when the TU size flag is 0 may be asmaller value from among the maximum transformation unit size and thecurrent prediction unit size.

If a prediction mode of a current partition unit is an intra mode,RootTuSize may be determined by using Equation (3) below. In Equation(3), PartitionSize denotes the size of the current partition unit.

RootTuSize=min(MaxTransformSize, PartitionSize) (3).

That is, if the current prediction mode is the intra mode, thetransformation unit size RootTuSize when the TU size flag is 0 may be asmaller value from among the maximum transformation unit size and thesize of the current partition unit.

However, the current maximum transformation unit size RootTuSize thatvaries according to the type of a prediction mode in a partition unit ismerely exemplary, and is not limited thereto in another exemplaryembodiment.

FIG. 14 is a flowchart illustrating a video encoding method according toan exemplary embodiment. Referring to FIG. 14, in operation 1210, acurrent picture is split into at least one maximum coding unit. Amaximum depth indicating a total number of possible splitting times maybe predetermined.

In operation 1220, a coded depth to output a final encoding resultaccording to at least one split region, which is obtained by splitting aregion of each maximum coding unit according to depths, is determined byencoding the at least one split region, and a coding unit according to atree structure is determined.

The maximum coding unit is spatially split whenever the depth deepens,and thus is split into coding units of a lower depth. Each coding unitmay be split into coding units of another lower depth by being spatiallysplit independently from adjacent coding units. Encoding is repeatedlyperformed on each coding unit according to depths.

Also, a transformation unit according to partition types having theleast encoding error is determined for each deeper coding unit. In orderto determine a coded depth having a minimum encoding error in eachmaximum coding unit, encoding errors may be measured and compared in alldeeper coding units according to depths.

In operation 1230, encoded image data that is the final encoding resultaccording to the coded depth is output for each maximum coding unit,with encoding information about the coded depth and an encoding mode.The information about the encoding mode may include at least one ofinformation about a coded depth or split information, information abouta partition type of a prediction unit, a prediction mode, and a size ofa transformation unit. The encoded information about the encoding modemay be transmitted to a decoder with the encoded image data.

FIG. 15 is a flowchart illustrating a video decoding method according toan exemplary embodiment. Referring to FIG. 15, in operation 1310, abitstream of an encoded video is received and parsed.

In operation 1320, encoded image data of a current picture assigned to amaximum coding unit and information about a coded depth and an encodingmode according to maximum coding units are extracted from the parsedbitstream. The coded depth of each maximum coding unit is a depth havingthe least encoding error in each maximum coding unit. In encoding eachmaximum coding unit, the image data is encoded based on at least onedata unit obtained by hierarchically splitting each maximum coding unitaccording to depths.

According to the information about the coded depth and the encodingmode, the maximum coding unit may be split into coding units having atree structure. Each of the coding units having the tree structure isdetermined as a coding unit corresponding to a coded depth, and isoptimally encoded as to output the least encoding error. Accordingly,encoding and decoding efficiency of an image may be improved by decodingeach piece of encoded image data in the coding units after determiningat least one coded depth according to coding units.

In operation 1330, the image data of each maximum coding unit is decodedbased on the information about the coded depth and the encoding modeaccording to the maximum coding units. The decoded image data may bereproduced by a reproducing apparatus, stored in a storage medium, ortransmitted through a network.

Hereinafter, video encoding and decoding performed in an operating modeof a coding tool considering a size of a coding unit according toexemplary embodiments will be described with reference to FIGS. 16 to23.

FIG. 16 is a block diagram of a video encoding apparatus 1400 based on acoding tool considering the size of a coding unit, according to anexemplary embodiment. Referring to FIG. 16, the apparatus 1400 includesa maximum coding unit splitter 1410, a coding unit determiner 1420, andan output unit 1430.

The maximum coding unit splitter 1410 splits a current picture into atleast one maximum coding unit.

The coding unit determiner 1420 encodes the at least one maximum codingunit in coding units corresponding to depths. In this case, the codingunit determiner 1420 may encode a plurality of split regions of the atleast one maximum coding unit in operating modes corresponding to codingtools according to the depths of the coding units, respectively, basedon a relationship between a depth of a coding unit, a coding tool, andan operating mode.

The coding unit determiner 1420 encodes coding units corresponding toall depths, compares the results of encoding with one another, anddetermines a depth of a coding unit having a highest coding efficiencyas a coded depth. Since in the split regions of the at least one maximumcoding unit, a depth having a highest coding efficiency may differaccording to location, a coded depth of each of the split regions of theat least one maximum coding unit may be determined independently ofthose of the other regions. Thus, more than one coded depth may bedefined in one maximum coding unit.

Examples of a coding tool for encoding may include quantization,transformation, intra prediction, inter prediction, motion compensation,entropy coding, and loop filtering, which are video encoding techniques.According to an exemplary embodiment, in the video encoding apparatus1400, each of a plurality of coding tools may be performed according toat least one operating mode. Here, the term, operating mode indicates amanner in which a coding tool is performed.

For example, if a coding tool is inter prediction, an operating mode ofthe coding tool may be classified into a first operating mode in which amedian value of motion vectors of neighboring prediction units isselected, a second operating mode in which a motion vector of aprediction unit at a particular location from among neighboringprediction units is selected, and a third operating mode in which amotion vector of a prediction unit that includes a template most similarto a template of a current prediction unit from among neighboringprediction units is selected.

According to an exemplary embodiment, the video encoding apparatus 1400may variably set an operating mode of a coding tool according to thesize of a coding unit. In the present exemplary embodiment, the videoencoding apparatus 1400 may variably set an operating mode of at leastone coding tool according to the size of a coding unit. Since a depth ofa coding unit corresponds to the size of the coding unit, the operatingmode of at least one coding tool may be determined based on the depth ofthe coding unit corresponding to the size of the coding unit. Thus, therelationship among a depth of a coding unit, a coding tool, and anoperating mode may be set. Similarly, if a coding tool may be performedin a prediction unit or a partition of a coding unit, an operating modeof the coding tool may be determined based on the size of a predictionunit or a partition.

The video encoding apparatus 1400 may set the relationship among a depthof a coding unit, a coding tool, and an operating mode before encodingis performed. For example, according to another exemplary embodiment,the video encoding apparatus 1400 may set the relationship among a depthof a coding unit, a coding tool, and an operating mode by encoding thecoding units of the at least one maximum coding unit corresponding todepths in all operating modes of a predetermined coding tool anddetecting an operating mode having a highest coding efficiency fromamong the operating modes.

The video encoding apparatus 1400 may assign an operating mode causingoverhead bits to coding units corresponding to depths, the sizes ofwhich are equal to or greater than a predetermined size, and may assignan operating mode that does not cause overhead bits to the other codingunits, the sizes of which are less than the predetermined size.

The video encoding apparatus 1400 may encode and transmit informationregarding the relationship among a depth of a coding unit, a codingtool, and an operating mode in slice units, frame units, picture units,or GOP units of an image. According to another exemplary embodiment, thevideo encoding apparatus 1400 may insert the information regardingencoding and the information regarding the relationship among a depth ofa coding unit, a coding tool, and an operating mode into an SPS.

If the coding unit determiner 1420 performs intra prediction, which is atype of a coding tool, an operating mode of intra prediction may beclassified according to a number of directions of prediction, i.e.,directions in which neighborhood information may be referred to. Thus,an operating mode of intra prediction performed by the video encodingapparatus 1400 may include intra prediction modes representing thenumber of directions of prediction that vary according to the size of acoding unit.

Also, if the coding unit determiner 1420 performs intra prediction, anoperating mode of intra prediction may be classified according towhether smoothing is to be performed in consideration of an imagepattern. Thus, an operating mode of intra prediction performed by thevideo encoding apparatus 1400 may represent whether intra prediction isto be performed according to the size of a coding unit bydifferentiating an intra prediction mode for smoothing a region of acoding unit and an intra prediction mode for retaining a boundary linefrom each other.

If the coding unit determiner 1420 performs inter prediction, which isanother type of a coding tool, the coding unit determiner 1420 mayselectively perform at least one method of determining a motion vector.Thus, an operating mode of inter prediction performed by the videoencoding apparatus 1400 may include an inter prediction moderepresenting a method of determining a motion vector, which isselectively performed according to the size of a coding unit.

If the coding unit determiner 1420 performs transformation, which isanother type of a coding tool, the coding unit determiner 1420 mayselectively perform rotational transformation according to the patternof an image. The coding unit determiner 1420 may store a matrix ofrotational transformation to be multiplied by a predetermined sized datamatrix, which is a transformation target, so as to effectively performrotational transformation. Thus, an operating mode of transformationperformed by the video encoding apparatus 1400 may include atransformation mode representing an index of a matrix of rotationaltransformation corresponding to the size of a coding unit.

If the coding unit determiner 1420 performs quantization, which isanother type of a coding tool, then a quantization parameter deltarepresenting a difference between a current quantization parameter and apredetermined representative quantization parameter may be used. Thus,an operating mode of quantization performed by the video encodingapparatus 1400 may include a quantization mode indicating whether thequantization parameter delta that varies according to the size of acoding unit is to be used.

If the coding unit determiner 1420 performs interpolation, which isanother type of a coding tool, interpolation filter may be used. Thecoding unit determiner 1420 may selectively set coefficients or thenumber of taps of the interpolation filter based on the size of a codingunit, a prediction unit or a partition and the depth of a coding unit.Thus, an operating mode of interpolation filtering performed by thevideo encoding apparatus 1400 may include an interpolation modeindicating coefficients or the number of taps of an interpolation filterthat varies according to the size or the depth of a coding unit and thesize of a prediction unit or a partition.

The output unit 1430 may output a bitstream, in which encoded video data(i.e., a final result of encoding received from the coding unitdeterminer 1420), information regarding a coded depth, and an encodingmode are included in for each of the at least one maximum coding unit.The encoded video data may be a set of a plurality of pieces of videodata that are encoded in coding units corresponding to coded depths ofthe split regions of the at least one maximum coding unit, respectively.

Also, the above operating modes of coding tools for coding unitscorresponding to depths may be encoded in the form of the informationregarding the relationship among a depth of a coding unit, a codingtool, and an operating mode and then be inserted into a bitstream.

According to an exemplary embodiment, the video encoding apparatus 1400may perform a coding tool, such as quantization, transformation, intraprediction, inter prediction, motion compensation, entropy encoding, andloop filtering. These coding tools may be performed in differentoperating modes in coding units corresponding to depths, respectively.The above operating modes are just illustrative examples given forconvenience of explanation, and the relationship between a depth of acoding unit (or the size of a coding unit), a coding tool, and anoperating mode in the video encoding apparatus 1400 is not limited tothe above exemplary embodiments.

FIG. 17 is a block diagram of a video decoding apparatus 1500 based on acoding tool considering a size of a coding unit, according to anexemplary embodiment. Referring to FIG. 17, the video decoding apparatus1500 includes a receiver 1510, an extractor 1520, and a decoder 1330.

The receiver 1510 receives and parses a bitstream including encodedvideo data. The extractor 1520 extracts the encoded video data,information regarding encoding, and information regarding a relationshipamong a depth of a coding unit, a coding tool, and an operating modefrom the bitstream received via the receiver 1510.

The encoded video data is obtained by encoding image data in maximumcoding units. The image data in each of the maximum coding units ishierarchically split into a plurality of split regions according depths,and each of the split regions is encoded in a coding unit of acorresponding coded depth. The information regarding encoding includesinformation regarding coded depths of the maximum coding units and anencoding mode.

For example, the information regarding the relationship among a depth ofa coding unit, a coding tool, and an operating mode may be set in imagedata units, e.g., maximum coding units, frame units, field units, sliceunits, or GOP units. In another example, the information regardingencoding, and the information regarding the relationship among a depthof a coding unit, a coding tool, and an operating mode may be extractedfrom an SPS. Image data encoded in coding units of image data may bedecoded in a selective operating mode of a coding tool, based on theinformation regarding the relationship among a depth of a coding unit, acoding tool, and an operating mode, which is defined in predeterminedunits of image data.

The decoder 1530 may decode the encoded video data in maximum codingunits and in operating modes of coding tools in coding unitscorresponding to at least one coded depth, respectively, based on theinformation regarding encoding and the information regarding therelationship among a depth of a coding unit, a coding tool, and anoperating mode that are extracted by the extractor 1520. The operatingmode of a coding tool may be set according to a size of a coding unit.Since a size of a coding unit corresponding to the coded depthcorresponds to the coded depth, the operation mode of the coding toolfor the coding unit corresponding to the coded depth may be determinedbased on the coded depth. Similarly, if the coding tool for the codingunit is performed based on a prediction unit or a partition of thecoding unit, the operation mode of the coding tool may be determinedbased on the size of a prediction unit or a partition.

Even if the relationship among a depth of a coding unit, a coding tool,and an operating mode is set according to a coding tool, the decoder1530 may perform a decoding tool corresponding to the coding tool. Forexample, the decoder 1530 may inversely quantize a bitstream in a codingunit corresponding to a coded depth, based on information regarding arelationship among a depth of a coding unit, quantization, and anoperating mode.

If the decoder 1530 performs intra prediction, which is a type of adecoding tool, the decoder 1530 may perform intra prediction on acurrent coding unit corresponding to a coded depth, based on informationregarding a relationship among a depth of a coding unit, intraprediction, and an intra prediction mode. For example, the decoder 1530may perform intra prediction on the current coding unit corresponding tothe coded depth based on the information regarding the relationshipamong a depth of a coding unit, intra prediction, and an intraprediction mode, and neighborhood information according to a number ofdirections of intra prediction corresponding to the size of the currentcoding unit.

Also, the decoder 1530 may determine whether to perform intra predictionaccording to the coded unit of the current coding unit bydifferentiating an intra prediction mode for smoothing and an intraprediction mode for retaining a boundary line from each other, based onthe information regarding the relationship among a depth of a codingunit, intra prediction, and an intra prediction mode.

If the decoder 1530 performs inter prediction, which is another type ofa decoding tool, the decoder 1530 may perform inter prediction on thecurrent coding unit corresponding to the coded depth based on theinformation regarding the relationship among a depth of a coding unit,inter prediction, and an inter prediction mode. For example, the decoder1530 may perform the inter prediction mode on the current coding unit ofthe coded depth by using a method of determining a motion vector, basedon the information regarding the relationship among a depth of a codingunit, inter prediction, and the inter prediction mode.

If the decoder 1530 performs inverse transformation, which is anothertype of a decoding tool, the decoder 1530 may selectively performinverse rotational transformation based on information regarding arelationship among a depth of a coding unit, transformation, and atransformation mode. Thus, the decoder 1530 may perform inverserotational transformation on the current coding unit corresponding tothe coded depth by using a matrix of rotational transformation of anindex corresponding to the coded depth, based on information regardingthe relationship among a depth of a coding unit, transformation, and theinverse transformation mode.

If the decoder 1530 performs inverse quantization, which is another typeof a coding tool, the decoder 1530 may perform inverse quantization onthe current coding unit corresponding to the coded depth by using aquantization parameter delta corresponding to the coded depth, based oninformation regarding a depth of a coding unit, quantization, and aquantization mode.

If the decoder 1530 performs interpolation or extrapolation, which isanother type of a coding tool, a filter for interpolation orextrapolation may be used. The decoder 1530 may perform filtering usingthe filter for interpolation or extrapolation for a current coding unitcorresponding to the coded depth, by using coefficients or the number oftaps of the filter for interpolation or extrapolation based on operatingmode of filtering for interpolation or extrapolation, indicatingcoefficients or the number of taps of the filter for interpolation orextrapolation. The operating mode of filtering for interpolation orextrapolation may correspond to at least one of the size of the currentcoding unit, and the size of a prediction unit or a partition of thecurrent coding unit.

The video decoding apparatus 1500 may reconstruct the original imagefrom image data decoded by the decoder 1530. The reconstructed image maybe reproduced by a display apparatus (not shown) or may be stored in astorage medium (not shown).

In the video encoding apparatus 1400 and the video decoding apparatus1500 according to exemplary embodiments, the size of a coding unit mayvary according to the characteristics of an image and a codingefficiency of the image. The size of a data unit, such as a coding unit,a prediction unit, or a transformation unit, may be increased so as toencode a large amount of image data, e.g., a high-resolution orhigh-quality image. The size of a macroblock having a hierarchicalstructure according to the H.264 standards may be 4×4, 8×8, or 16×16,but the video encoding apparatus 1400 and the video decoding apparatus1500 according to one or more exemplary embodiments may expand the sizeof a data unit to 4×4, 8×8, 16×16, 32×32, 64×64, 128×128, or more.

The larger a data unit, the more image data included in the data unit,and the more various image characteristics in data units. Thus, it wouldbe inefficient to encode all data units having various sizes by usingonly one coding tool.

Accordingly, the video encoding apparatus 1400 may determine a depth ofa coding unit and an operating mode of a coding tool according to thecharacteristics of image data so as to increase a coding efficiency andencode information regarding a relationship among the depth of thecoding unit, the coding tool, and the operating mode. Furthermore, thevideo decoding apparatus 1500 may reconstruct the original image bydecoding a received bitstream, based on the information regarding arelationship among the depth of the coding unit, the coding tool, andthe operating mode.

Accordingly, the video encoding apparatus 1400 and the video decodingapparatus 1500 may effectively encode and decode a large amount of imagedata, such as a high-resolution or high-quality image, respectively.

FIG. 18 is a diagram for describing a relationship among the size of acoding unit, a coding tool, and an operating mode, according to anexemplary embodiment.

Referring to FIG. 18, according to an exemplary embodiment, in a videoencoding apparatus 1400 or a video decoding apparatus 1500, a 4×4 codingunit 1610, an 8×8 coding unit 1620, a 16×16 coding unit 1630, a 32×32coding unit 1640, and 64×64 coding unit 1650 may be used as codingunits. If a maximum coding unit is the 64×64 coding unit 1650, a depthof the 64×64 coding unit 1650 is 0, a depth of the 32×32 coding unit1640 is 1, a depth of the 16×16 coding unit 1630 is 2, a depth of the8×8 coding unit 1620 is 3, and a depth of the 4×4 coding unit 1610 is 4.

The video encoding apparatus 1400 may adaptively determine an operatingmode of a coding tool according to a depth of a coding unit. Forexample, if a first coding tool TOOL1 may be performed in a firstoperating mode TOOL1-1 1660, a second operating mode TOOL1-2 1662, and athird operating mode TOOL1-3, the video encoding apparatus 1400 mayperform the first coding tool TOOL1 in the first operating mode TOOL1-11660 with respect to the 4×4 coding unit 1610 and the 8×8 coding unit1620, perform the first coding tool TOOL1 in the second operating mode1662 with respect to the 16×16 coding unit 1630 and the 32×32 codingunit 1640, and perform the first coding tool TOOL1 in the thirdoperating mode 1664 with respect to the 64×64 coding unit 1650.

The relationship among the size of a coding unit, a coding tool, and anoperating mode may be determined by encoding a current coding unit inall operating modes of a corresponding coding tool and detecting anoperating mode causing a result of encoding with a highest codingefficiency from among the operating modes, during encoding of thecurrent coding unit. In another exemplary embodiment, the relationshipamong the size of a coding unit, a coding tool, and an operating modemay be predetermined by, for example, at least one of the performance ofan encoding system, a user's requirements, or ambient conditions.

Since the size of a maximum coding unit is fixed with respect topredetermined data, the size of a coding unit corresponds to a depth ofthe coding unit itself. Thus, a relationship between a coding tooladaptive to the size of a coding unit and an operating mode may beencoded by using information regarding a relationship among a depth of acoding unit, a coding tool, and an operating mode.

The information regarding the relationship among a depth of a codingunit, a coding tool, and an operating mode may indicate optimaloperating modes of coding tools in units of depths of coding units,respectively.

TABLE 2 Depth of Depth of Depth of Depth of Depth of coding unit = 4coding unit = 3 coding unit = 2 coding unit = 1 coding unit = 0operating first operating first operating second second third mode offirst mode mode operating operating operating coding tool mode mode modeoperating first operating second second third third mode of modeoperating operating operating operating second mode mode mode modecoding tool

According to exemplary Table 2, the operating modes of the first andsecond coding tools may be variable applied to coding units havingdepths 4, 3, 2, 1, and 0, respectively. The information regarding therelationship among a depth of a coding unit, a coding tool, and anoperating mode may be encoded and transmitted in sequence units, GOPunits, picture units, frame units, or slice units of an image.

Various exemplary embodiments of a relationship among a depth of acoding unit, a coding tool, and an operating mode will now be describedin detail.

FIG. 19 is a diagram for describing a relationship among a depth of acoding unit, a coding tool (e.g., inter prediction), and an operatingmode, according to an exemplary embodiment.

If a video encoding apparatus 1400 according to an exemplary embodimentperforms inter prediction, at least one method of determining a motionvector may be used. Thus, an operating mode of inter prediction, whichis a type of a coding tool, may be classified according to a method ofdetermining a motion vector.

For example, referring to FIG. 19, in a first operating mode of interprediction, a median value of motion vectors mvpA, mvpB, and mvpC ofneighboring coding units A, B, and C 1710, 1720, and 1730 is selected asa predicted motion vector MVP of a current coding unit 1700, asindicated in Equation (4) below:

MVP=median(mvpA, mvpB, mvpC)   (4).

If the first operating mode is employed, an amount of calculation is lowand overhead bits may not be used. Thus, even if inter prediction isperformed on small-sized coding units in the first operating mode, anamount of calculation or an amount of bits to be transmitted is small.

For example, in a second operating mode of inter prediction, an index ofthe motion vector of a coding unit that is selected as a predictedmotion vector of the current coding unit 1700 from among the motionvectors of the neighboring coding units A, B, and C 1710, 1720, and1730, is displayed directly.

For example, if the video encoding apparatus 1400 performs interprediction on the current coding unit 1700, the motion vector mvpA ofthe neighboring coding unit A 1710 may be selected as an optimalpredicted motion vector of the current coding unit 1700 and an index ofthe motion vector mvpA may be encoded. Thus, although overhead occurs inan encoding side, caused by an index representing the predicted motionvector, an amount of calculation when performing inter prediction in thesecond operating mode is small in a decoding side.

For example, in a third operating mode of inter prediction, pixels 1705on a predetermined location on the current coding unit 1700 are comparedwith pixels 1715, 1725, 1735 on predetermined locations on theneighboring coding units A, B, and C 1710, 1720, and 1730, pixels, thedistortion degrees of which are lowest are detected from among thepixels 1715, 1725, 1735, and a motion vector of a neighboring codingunit including the detected pixels is selected as a predicted motionvector of the current coding unit 1700.

Thus, although an amount of calculation may be large for the decodingside to detect pixels, the distortion degrees of which are lowest, theencoding side does not experience overhead in bits to be transmitted. Inparticular, if inter prediction is performed on an image sequenceincluding a specific image pattern in the third operating mode, a resultof prediction is more precise than when a median value of motion vectorsof neighboring coding units is used.

The video encoding apparatus 1400 may encode information regarding arelationship among the first operating mode, the second operating mode,and the third operating mode of inter prediction determined according toa depth of a coding unit. The video decoding apparatus 1500 according toan exemplary embodiment may decode image data by extracting theinformation regarding the first operating mode, the second operatingmode, and the third operating mode of inter prediction determinedaccording to the depth of the coding unit, from a received bitstream,and performing a decoding tool related to motion compensation and interprediction performed on a current coding unit of a coded depth, based onthe extracted information.

The video encoding apparatus 1400 checks whether overhead occurs in bitsto be transmitted so as to determine an operating mode of interprediction according to a size or depth of a coding unit. If a smallcoding unit is encoded, additional overhead may greatly lower a codingefficiency thereof, whereas if a large coding unit is encoded, a codingefficiency is not significantly influenced by additional overhead.

Accordingly, it may be efficient to perform inter prediction in thethird operating mode that does not cause additional overhead when asmall coding unit is encoded. In this regard, an example of arelationship between the size of a coding unit and an operating mode ofinter prediction is shown in exemplary Table 3 below:

TABLE 3 Size of Size of Size of Size of Size of coding coding codingcoding coding unit = unit = 4 unit = 8 unit = 16 unit = 32 64 operatingthird third first second second mode of operating operating operatingoperating operating inter mode mode mode mode mode prediction

FIG. 20 is a diagram for describing a relationship among a depth of acoding unit, a coding tool (e.g., intra prediction), and an operatingmode, according to an exemplary embodiment.

A video encoding apparatus 1400 according to an exemplary embodiment mayperform directional extrapolation as intra prediction by usingreconstructed pixels 1810 neighboring to a current coding unit 1800. Forexample, a direction of intra prediction may be defined as tan⁻¹(dx,dy), and inter prediction may be performed in various directionsaccording to a plurality of (dx, dy) parameters.

A neighboring pixel 1830 on a line extending from a current pixel 1820in the current coding unit 1800, which is to be predicted, and beinginclined by an angle of tan⁻¹(dy/dx) determined by values dx and dy fromthe current pixel 1820, may be used as a predictor of the current pixel1830. The neighboring pixel 1830 may belong to a coding unit that islocated to an upper or left side of the current coding unit 1800, whichwas previously encoded and reconstructed.

If intra prediction is performed, the video encoding apparatus 1400 mayadjust a number of directions of intra prediction according to the sizeof a coding unit. Thus, operating modes of intra prediction, which is atype of a coding tool, may be classified according to the number of thedirections of intra prediction.

A number of directions of intra prediction may vary according to thesize and hierarchical tree structure of a coding unit. Overhead bitsused to represent an intra prediction mode may decrease a codingefficiency of a small coding unit but does not affect a codingefficiency of a large coding unit.

Thus, the video encoding apparatus 1400 may encode information regardinga relationship among a depth of a coding unit and the number ofdirections of intra prediction. Also, a video decoding apparatus 1500according to an exemplary embodiment may decode image data by extractingthe information regarding a relationship among a depth of a coding unitand the number of directions of intra prediction from a receivedbitstream, and performing a decoding tool related to intra predictionperformed on a current coding unit of a coded depth, based on theextracted information.

The video encoding apparatus 1400 considers an image pattern of thecurrent coding unit so as to determine an operating mode of intraprediction according to the size or depth of a coding unit. In the caseof an image containing detailed components, intra prediction may beperformed by using linear extrapolation, and thus, a large number ofdirections of intra prediction may be used. However, in the case of aflat region of an image, the number of directions of intra predictionmay be relatively small. For example, a plain mode or a bi-linear modeusing interpolation of reconstructed neighboring pixels may be used toperform intra prediction on a flat region of an image.

Since a large coding unit is probably determined in a flat region of animage, the number of directions of intra prediction may be relativelysmall when an intra prediction mode is performed on the large codingunit. Also, since a small coding unit is probably determined in a regionincluding detailed components of an image, the number of directions ofintra prediction may be relatively large when the intra prediction modeis performed on the small coding unit. Thus, a relationship between thesize of a coding unit and the intra prediction mode may be considered asa relationship between the size of the coding unit and the number ofdirections of intra prediction. An example of the relationship betweenthe size of the coding unit and the number of directions of intraprediction is shown in exemplary Table 4 below:

TABLE 4 Size of Size of Size of Size of Size of coding coding codingcoding coding unit = 4 unit = 8 unit = 16 unit = 32 unit = 64 Number of9 9 33 17 5 directions of intra prediction

A large coding unit may include image patterns that are arranged invarious directions, and intra prediction may thus be performed on thelarge coding unit by using linear extrapolation. In this case, arelationship between the size of a coding unit and the intra predictionmode may be set as shown in exemplary Table 5 below:

TABLE 5 Size of Size of Size of Size of Size of coding coding codingcoding coding unit = 4 unit = 8 unit = 16 unit = 32 unit = 64 Number of9 9 33 33 17 directions of intra prediction

According to an exemplary embodiment, prediction encoding is performedin various intra prediction modes set according to the sizes of codingunits, thereby more efficiently compressing an image according to thecharacteristics of the image.

Predicted coding units output from the video encoding apparatus 1400 byperforming various intra prediction modes according to depths of codingunits have a predetermined directionality according to the type of anintra prediction mode. Due to a directionality in such predicted codingunits, an efficiency of predicting may be high when pixels of a currentcoding unit that is to be encoded have a predetermined directionality,and may be low when the pixels of the current coding unit do not havethe predetermined orientation. Thus, a predicted coding unit obtainedusing intra prediction may be post-processed by producing a newpredicted coding unit by changing values of pixels in the predictedcoding unit by using these pixels and at least one neighboring pixel,thereby improving an efficiency of predicting an image.

For example, in the case of a flat region of an image, it may beefficient to perform post-processing for smoothing on a predicted codingunit obtained using intra prediction. Also, in the case of a regionhaving detailed components of the image, it may be efficient to performa post-processing for retaining the detailed components on a predictedcoding unit obtained using intra prediction.

Thus, the video encoding apparatus 1400 may encode information regardinga relationship between a depth of a coding unit and an operating modeindicating whether a predicted coding unit obtained using intraprediction is to be post-processed. Also, the video decoding apparatus1500 may decode image data by extracting the information regarding therelationship between a depth of a coding unit and an operating modeindicating whether a predicted coding unit obtained using intraprediction is to be post-processed, from a received bitstream, andperforming a decoding tool related to intra prediction performed on acurrent coding unit of a coded depth, based on the extractedinformation.

In the video encoding apparatus 1400, an intra prediction mode, in whichpost-processing for smoothing is performed and an intra prediction modein which post-processing for smoothing is not performed, may be selectedfor a flat region of an image and a region including detailed componentsof the image, respectively, as the operating mode indicating whether apredicted coding unit obtained using intra prediction is to bepost-processed.

A large coding unit may be determined in a flat region of an image and asmall coding unit may be determined in a region containing detailedcomponents of the image. Thus, the video encoding apparatus 1400 maydetermine that an intra prediction mode, in which post-processing forsmoothing is performed, is performed on the large coding unit and anintra prediction mode, in which post-processing for smoothing is notperformed, is performed on the small coding unit.

Accordingly, a relationship between a depth of a coding unit and anoperating mode indicating whether a predicted coding unit obtained byintra prediction is to be post-processed may be considered as arelationship between the size of a coding unit and whetherpost-processing is to be performed. In this regard, an example of arelationship among the size of a coding unit and an operating mode ofintra prediction may be shown in exemplary Table 6 below:

TABLE 6 Size of Size of Size of Size of Size of coding coding codingunit = coding unit = coding unit = unit = 4 unit = 8 16 32 64 Post- 0 01 1 1 processing mode of intra prediction

If the video encoding apparatus 1400 performs transformation, which is atype of a coding tool, rotational transformation may be selectivelyperformed according to an image pattern. For efficient calculation ofrotational transformation, a data matrix for rotational transformationmay be stored in memory. If the video encoding apparatus 1400 performsrotational transformation or if the video decoding apparatus 1500performs inverse rotational transformation, related data may be calledfrom the memory by using an index of rotational transformation data usedfor the calculation. Such rotational transformation data may be set incoding units or transformation units, or according to the type of asequence.

Thus, the video encoding apparatus 1400 may set a transformation modeindicated by an index of a matrix of rotational transformationcorresponding to a depth of a coding unit as an operating mode oftransformation. The video encoding apparatus 1400 may encode informationregarding a relationship between the size of a coding unit and thetransformation mode indicating the index of the matrix of rotationaltransformation.

The video decoding apparatus 1500 may decode image data by extractingthe information regarding the relationship between a depth of a codingunit and the transformation mode indicating the index of the matrix ofrotational transformation from a received bitstream, and performinginverse rotational transformation on a current coding unit of a codeddepth, based on the extracted information.

Accordingly, a relationship among a depth of a coding unit, rotationaltransformation, and an operating mode may be considered as arelationship between the size of a coding unit and the index of thematrix of rotational transformation. In this regard, a relationshipbetween the size of a coding unit and an operating mode of rotationaltransformation may be shown in exemplary Table 7 below:

TABLE 7 Size of Size of Size of Size of Size of coding coding codingcoding coding unit = unit = 4 unit = 8 unit = 16 unit = 32 64 Index of4-7 4-7 0-3 0-3 0-3 matrix of rotational transformation

If the video encoding apparatus 1400 performs quantization, which is atype of a coding tool, a quantization parameter delta representing adifference between a current quantization parameter and a predeterminedrepresentative quantization parameter may be used. The quantizationparameter delta may vary according to the size of a coding unit. Thus,in the video encoding apparatus 1400, an operating mode of quantizationmay include a quantization mode indicating whether the quantizationparameter delta varying according to the size of a coding unit is to beused.

Thus, the video encoding apparatus 1400 may set a quantization modeindicating whether the quantization parameter delta corresponding to thesize of a coding unit is to be used as an operating mode ofquantization. The video encoding apparatus 1400 may encode informationregarding a relationship between a depth of a coding unit and thequantization mode indicating whether the quantization parameter delta isto be used.

The video decoding apparatus 1500 may decode image data by extractingthe information regarding the relationship between a depth of a codingunit and the quantization mode indicating whether the quantizationparameter delta is to be used, from a received bitstream, and performinginverse quantization on a current coding unit of a coded depth, based onthe extracted information.

Accordingly, a relationship among a depth of a coding unit,quantization, and an operating mode may be considered as a relationshipbetween the size of a coding unit and whether the quantization parameterdelta is to be used. In this regard, an example of a relationshipbetween the size of a coding unit and an operating mode of quantizationis as shown in exemplary Table 8 below:

TABLE 8 Size of Size of Size of Size of Size of coding coding codingcoding unit = coding unit = 4 unit = 8 unit = 16 32 unit = 64Quantization false false true false false parameter delta

FIG. 21 illustrates syntax of a sequence parameter set 1900, in whichinformation regarding a relationship among a depth of a coding unit, acoding tool, and an operating mode is inserted, according to anexemplary embodiment.

In FIG. 21, sequence_parameter set denotes syntax of the sequenceparameter set 1900 for a current slice. Referring to FIG. 21, theinformation regarding the relationship among a depth of a coding unit, acoding tool, and an operating mode is inserted into the syntax of thesequence parameter set 1900 for the current slice.

Furthermore, in FIG. 21, picture_width denotes the width of an inputimage, picture_height denotes the height of the input image,max_coding_unit_size denotes the size of a maximum coding unit, andmax_coding_unit_depth denotes a maximum depth.

According to an exemplary embodiment, syntaxesuse_independent_cu_decode_flag indicating whether decoding is to beindependently performed in coding units, use_independent_cu_parse_flagindicating whether parsing is to be independently performed in codingunits, use_my_accuracy_control_flag indicating whether a motion vectoris to be accurately controlled, use_arbitrary_direction_intra_flagindicating whether intra prediction is to be performed in an arbitrarydirection, use_frequency_domain_prediction_flag indicating whetherprediction encoding/decoding is to be performed in frequencytransformation domain, use_rotational_transform_flag indicating whetherrotational transformation is to be performed,use_tree_significant_map_flag indicating whether encoding/decoding is tobe performed using a tree significant map,use_multi_parameter_intra_prediction_flag indicating whether intraprediction encoding is to be performed using a multi parameter,use_advanced_motion_vector_prediction flag indicating whether advancedmotion vector prediction is to be performed,use_adaptive_loop_filter_flag indicating whether adaptive loop filteringis to be performed, use_quadtree_adaptive_loop_filter_flag indicatingwhether quadtree adaptive loop filtering is to be performed,use_delta_qp_flag indicating whether quantization is to be performedusing a quantization parameter delta, use_random_noise_generation_flagindicating whether random noise generation is to be performed,use_asymmetric_motion_partition_flag indicating whether motionestimation is to be performed in asymmetric prediction units, may beused as examples of a sequence parameter of a slice. It is possible toefficiently encode or decode the current slice by setting whether theabove operations are to be used by using these syntaxes.

In particular, the length of an adaptive loop filter alf_filter_length,the type of the adaptive loop filter alf_filter_type, a reference valuefor quantizing an adaptive loop filter coefficient alf_qbits, and thenumber of color components of adaptive loop filtering alf_num_color maybe set in the sequence parameter set 1900, based onuse_adaptive_loop_filter_flag anduse_quadtree_adaptive_loop_filter_flag.

The information regarding the relationship among a depth of a codingunit, a coding tool, and an operating mode used in a video encodingapparatus 1400 and a video decoding apparatus 1500 according toexemplary embodiments may indicate an operating mode of inter predictioncorresponding to a depth of a coding unit uiDepth mvp_mode[uiDepth], andan operating mode significant_map_mode[uiDepth] indicating the type of asignificant map from among tree significant maps. That is, either arelationship between inter prediction and a corresponding operating modeaccording to a depth of a coding unit, or a relationship betweenencoding/decoding using the tree significant map and a correspondingoperating mode according to a depth of a coding unit, may be set in thesequence parameter set 1900.

A bit depth of an input sample input_sample_bit_depth and a bit depth ofan internal sample internal_sample_bit_depth may also be set in thesequence parameter set 1900.

Information regarding a relationship among a depth of a coding unit, acoding tool, and an operating mode encoded by the video encodingapparatus 1400 or decoded by the video decoding apparatus 1500 accordingto an exemplary embodiment is not limited to the information inserted inthe sequence parameter set 1900 illustrated in FIG. 21. For example, theinformation may be encoded or decoded in maximum coding units, sliceunits, frame units, picture units, or GOP units of the image.

FIG. 22 is a flowchart illustrating a video encoding method based on acoding tool considering a size of a coding unit, according to anexemplary embodiment. Referring to FIG. 22, in operation 2010, a currentpicture is split into at least one maximum coding unit.

In operation 2020, a coded depth is determined by encoding the at leastone maximum coding unit in coding units corresponding to depths inoperating modes of coding tools, respectively, based on a relationshipamong a depth of at least one coding unit of the at least one maximumcoding unit, a coding tool, and an operating mode. Thus, the at leastone maximum coding unit includes coding units corresponding to at leastone coded depth.

The relationship among a depth of at least one coding unit of the atleast one maximum coding unit, a coding tool, and an operating mode maybe preset in units of slices, frames, GOPs, or frame sequences of animage. The relationship among a depth of at least one coding unit of theat least one maximum coding unit, a coding tool, and an operating modemay be determined by comparing results of encoding the coding unitscorresponding to depths in at least one operating mode matching codingtools with one another, and selecting an operating mode having a highestcoding efficiency from among the at least one operating mode duringencoding of the at least one maximum coding unit. Otherwise, therelationship among a depth of at least one coding unit of the at leastone maximum coding unit, a coding tool, and an operating mode, may bedetermined in such a manner that coding units corresponding to depths,the sizes of which are less than or equal to a predetermined size, maycorrespond to an operating mode that does not cause overhead bits to beinserted in an encoded data stream and the other coding units, the sizesof which are greater than the predetermined size, may correspond to anoperating mode causing the overhead bits.

In operation 2030, a bitstream including encoded video data of the atleast one coded depth, information regarding encoding, and informationregarding the relationship among a depth of at least one coding unit ofthe at least one maximum coding unit, a coding tool, and an operatingmode in the at least one maximum coding unit is output. The informationregarding encoding may include the at least one coded depth andinformation regarding an encoding mode in the at least one maximumcoding unit. The information regarding the relationship among a depth ofat least one coding unit of the at least one maximum coding unit, acoding tool, and an operating mode, may be inserted in slice units,frame units, GOPs, or frame sequences of the image.

FIG. 23 is a flowchart illustrating a video decoding method based on acoding tool considering a size of a coding unit, according to anexemplary embodiment. Referring to FIG. 23, in operation 2110, abitstream including encoded video data is received and parsed.

In operation 2120, the encoded video data, information regardingencoding, and information regarding a relationship among a depth of acoding unit, a coding tool, and an operating mode are extracted from thebitstream. The information regarding a relationship among a depth of acoding unit, a coding tool, and an operating mode may be extracted fromthe bitstream in maximum coding units, slice units, frame units, GOPunits, or frame sequences of an image.

In operation 2130, the encoded video data is decoded in maximum codingunits according to an operating mode of a coding tool matching a codingunit corresponding to at least one coded depth, based on the informationregarding encoding and the information regarding a relationship among adepth of a coding unit, a coding tool, and an operating mode, extractedfrom the bitstream.

While not restricted thereto, one or more exemplary embodiments can bewritten as computer programs and can be implemented in general-usedigital computers that execute the programs using a computer readablerecording medium. Examples of the computer readable recording mediuminclude magnetic storage media (e.g., ROM, floppy disks, hard disks,etc.) and optical recording media (e.g., CD-ROMs, or DVDs). Moreover,while not required in all exemplary embodiments, one or more units ofthe video encoding apparatus 100 or 1400, the video decoding apparatus200 or 1500, the image encoder 400, and the image decoder 500 caninclude a processor or microprocessor executing a computer programstored in a computer-readable medium.

While exemplary embodiments have been particularly shown and describedwith reference to the drawings above, it will be understood by those ofordinary skill in the art that various changes in form and details maybe made therein without departing from the spirit and scope of theinventive concept as defined by the appended claims. The exemplaryembodiments should be considered in descriptive sense only and not forpurposes of limitation. Therefore, the scope of the inventive concept isdefined not by the detailed description of the exemplary embodiments butby the appended claims, and all differences within the scope will beconstrued as being included in the present inventive concept.

1. A method of encoding video data, the method comprising: splitting acurrent picture of the video data into at least one maximum coding unit;determining a coded depth to output an encoding result by encoding atleast one split region of the at least one maximum coding unit accordingto at least one operating mode of at least one coding tool,respectively, based on a relationship among a depth of at least onecoding unit of the at least one maximum coding unit, a coding tool, andan operating mode, wherein the at least one split region is generated byhierarchically splitting the at least one maximum coding unit accordingto depths and an operation mode of a coding tool for a coding unit isdetermined according to a depth of the coding unit; and outputting abitstream comprising encoded video data of the coded depth, informationregarding a coded depth of at least one maximum coding unit, informationregarding an encoding mode, and information regarding the relationshipamong the depth of the at least one coding unit of the at least onemaximum coding unit, the coding tool, and the operating mode in the atleast one maximum coding unit.
 2. The method of claim 1, wherein: the atleast one coding tool for the encoding of the at least one maximumcoding unit comprises at least of quantization, transformation, intraprediction, inter prediction, motion compensation, entropy encoding, andloop filtering; and the at least one coding tools are performed in atleast one operating mode, respectively.
 3. The method of claim 1,wherein the information regarding the relationship among the depth ofthe at least one coding unit of the at least one maximum coding unit,the coding tool, and the operating mode is preset in slice units, frameunits, or frame sequence units of the current picture.
 4. The method ofclaim 1, wherein the relationship among the depth of the at least onecoding unit of the at least one maximum coding unit, the coding tool,and the operating mode is determined by comparing results of encodingthe at least one maximum coding unit in the at least one operating modeof the at least one coding tools matching coding units according todepths with one another, and selecting an operating mode having ahighest coding efficiency from among the at least one operating mode,during the encoding of the at least one maximum coding unit.
 5. Themethod of claim 1, wherein the relationship among the depth of the atleast one coding unit of the at least one maximum coding unit, thecoding tool, and the operating mode is determined such that coding unitscorresponding to depths, sizes of which are less than or equal to apredetermined size, correspond to an operating mode that does not causeoverhead bits to be inserted into an encoded data stream and othercoding units, sizes of which are greater than the predetermined size,correspond to an operating mode that causes the overhead bits.
 6. Themethod of claim 1, wherein a coded depth of each of the at least onesplit region is determined independently of other split regions of thesame maximum coding unit to which the at least one split region belongs.7. The method of claim 1, wherein a sequence parameter set of thebitstream comprises at least one of the information regarding the codeddepth of the at least one maximum coding unit, the information regardingthe encoding mode, and the information regarding the relationship amongthe depth of the at least one coding unit of the at least one maximumcoding unit, the coding tool, and the operating mode.
 8. A method ofdecoding video data, the method comprising: receiving and parsing abitstream comprising encoded video data; extracting, from the bitstream,the encoded video data, information regarding a coded depth of at leastone maximum coding unit, information regarding an encoding mode, andinformation regarding a relationship among a depth of at least onecoding unit of the at least one maximum coding unit, a coding tool, andan operating mode; and decoding the encoded video data in the at leastone maximum coding unit according to an operating mode of a coding toolmatching a coding unit corresponding to at least one coded depth, basedon the information regarding the coded depth of the at least one maximumcoding unit, the information regarding the encoding mode, and theinformation regarding the relationship among the depth of the at leastone coding unit of the at least one maximum coding unit, the codingtool, and the operating mode, wherein the operation mode of the codingtool for a coding unit is determined according to the coded depth of thecoding unit.
 9. The method of claim 8, wherein: the coding tool for theencoding of the at least one maximum coding unit comprises at least oneof quantization, transformation, intra prediction, inter prediction,motion compensation, entropy encoding, and loop filtering; and thedecoding the encoded video data comprises performing a decoding toolcorresponding to the coding tool for the encoding of the at least onemaximum coding unit.
 10. The method of claim 8, wherein the informationregarding the relationship among the depth of the at least one codingunit of the at least one maximum coding unit, the coding tool, and theoperating mode is preset in slice units, frame units, or frame sequenceunits of the current picture.
 11. The method of claim 8, wherein therelationship among the depth of the at least one coding unit of the atleast one maximum coding unit, the coding tool, and the operating modeis determined by comparing results of encoding the at least one maximumcoding unit in at least one operating mode of at least one coding toolmatching coding units according to depths with one another, andselecting an operating mode having a highest coding efficiency fromamong the at least one operating mode during the encoding of the atleast one maximum coding unit.
 12. The method of claim 8, wherein therelationship among the depth of the at least one coding unit of the atleast one maximum coding unit, the coding tool, and the operating modeis determined such that coding units corresponding to depths, sizes ofwhich are less than or equal to a predetermined size, correspond to anoperating mode that does not cause overhead bits to be inserted into anencoded data stream and other coding units corresponding to depths,sizes of which are greater than the predetermined size, correspond to anoperating mode that causes the overhead bits, during encoding of thebitstream.
 13. The method of claim 8, wherein a coded depth of each ofat least one coding unit of the at least one maximum coding unit isdetermined independently of other coding units of the same maximumcoding unit to which the at least one coding unit belongs, wherein theat least one coding unit is generated by hierarchically splitting the atleast one maximum coding unit according to depths.
 14. The method ofclaim 8, wherein the extracting comprises extracting, from a sequenceparameter set of the bitstream, at least one of the informationregarding the coded depth of the at least one maximum coding unit, theinformation regarding the encoding mode, and the information regardingthe relationship among the depth of the at least one coding unit of theat least one maximum coding unit, the coding tool, and the operatingmode.
 15. The method of claim 8, wherein the coding tool is intraprediction, and the operation mode is one of an intra prediction modeclassified according to a number of directions of intra prediction, anintra prediction mode for smoothing regions in coding unitscorresponding to depths, and an intra prediction mode for retaining aboundary line.
 16. The method of claim 8, wherein the coding tool isinter prediction, and the operation mode is an inter prediction modeaccording to at least one method of determining a motion vector.
 17. Themethod of claim 8, wherein the coding tool is transformation, and theoperating mode is a transformation mode classified according to an indexof a matrix of rotational transformation.
 18. The method of claim 8,wherein the coding tool is quantization, and the operating mode is atquantization mode classified according to whether a quantizationparameter delta is to be used.
 19. An apparatus for encoding video data,the apparatus comprising: a maximum coding unit splitter which splits acurrent picture of the video data into at least one maximum coding unit;a coding unit determiner which determines a coded depth to output anencoding result by encoding at least one split region of the at leastone maximum coding unit according to at least one operating mode of atleast one coding tool, respectively, based on a relationship among adepth of at least one coding unit of the at least one maximum codingunit, a coding tool, and an operating mode, wherein the at least onesplit region is generated by hierarchically splitting the at least onemaximum coding unit according to depths and an operation mode of acoding tool for a coding unit is determined according to a depth of thecoding unit; and an output unit which outputs a bitstream comprisingencoded video data that is the encoding result, information regarding acoded depth of the at least one maximum coding unit, informationregarding an encoding mode, and information regarding the relationshipamong the depth of the at least one coding unit of the at least onemaximum coding unit, the coding tool, and the operating mode in the atleast one maximum coding unit.
 20. An apparatus for decoding video data,the apparatus comprising: a receiver which receives and parses abitstream comprising encoded video data; an extractor which extracts,from the bitstream, the encoded video data, information regarding acoded depth of at least one maximum coding unit, information regardingan encoding mode, and information regarding a relationship among a depthof at least one coding unit of the at least one maximum coding unit, acoding tool, and an operating mode; and a decoder which decodes theencoded video data in the at least one maximum coding unit according toan operating mode of a coding tool matching a coding unit correspondingto at least one coded depth, based on the information regarding thecoded depth of the at least one maximum coding unit, the informationregarding the encoding mode, and the information regarding therelationship among the depth of the at least one coding unit of the atleast one maximum coding unit, the coding tool, and the operating mode,wherein the operation mode of the coding tool for a coding unit isdetermined according to the coded depth of the coding unit.
 21. A methodof decoding video data, the method comprising: decoding encoded videodata in at least one maximum coding unit according to an operating modeof a coding tool matching a coding unit corresponding to at least onecoded depth, based on information regarding a coded depth of the atleast one maximum coding unit, information regarding an encoding mode,and information regarding a relationship among a depth of at least onecoding unit of the at least one maximum coding unit, a coding tool, andan operating mode, wherein the operation mode of the coding tool for acoding unit is determined according to the coded depth of the codingunit.
 22. A computer readable recording medium having recorded thereon aprogram for executing the method of claim
 1. 23. A computer readablerecording medium having recorded thereon a program for executing themethod of claim
 8. 24. A computer readable recording medium havingrecorded thereon a program for executing the method of claim 21.